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Preface 


This  study  investigates  the  assertion  that  the  Cost 
at  Completion  is  bounded  below  by  the  Cost  Performance 
Index-based  Estimate  at  Completion  (EAC)  and  above  by  the 
Schedule  Cost  Index-based  EAC.  The  hypothesis  is  tested 
on  a  selective  sample  of  321  contracts  from  the  Defense 
Acquisition  Executive  Summary  Database.  This  thesis 
checks  the  validity  of  this  assertion  and  explains  the 
usefulness  of  this  information - 

Special  thanks  to  Major  David  S.  Christensen  for  his 
advice  and  guidance  throughout  the  entire  thesis  process. 
Thanks  to  Captain  Timothy  J.  Halloran  for  his  assistance 
with  Paradox*  and  thanks  to  OUSD(A)  for  providing  the 
original  database  for  analysis. 


Mark  Terry  and  Mary  Vanderburgh 
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Abstract 

This  study  explores  the  widely  held  assertion  that  DOD 
contract  Cost  at  Completion  is  bounded  below  by  the  Cost 
Performance  Index-based  Estimate  at  Completion  (EAC)  and 
above  by  the  Schedule  Cost  Index-based  EAC. 

Descriptive  statistics  determined  the  floor  and 
ceiling  for  321  DOD  contracts.  The  results  confirmed  that 
the  Cost  Performance  Index-based  EAC  is  a  reasonable  floor 
and  the  Schedule  Cost  Index-based  EAC  is  a  reasonable 
ceiling  for  EAC  formulas.  For  the  contracts  considered 
overall,  on  average,  the  Cost  at  Completion  was  not  bounded 
by  the  floor  and  ceiling.  The  range  of  EAC  formulas 
evaluated  tended  to  slightly  underestimate  the  Cost  at 
Completion  on  average. 

Results  were  tested  for  sensitivity  to  Index  Type 
(cumulative,  six-month  and  three-month) ,  Program  Phase, 
Contract  Type,  Branch  of  Service,  System  Type,  Major 
Contract  Baseline  Changes  and  Management  Reserve. 

Graphs  of  the  EAC  ceilings  and  floors  for  several 
contract  categories  illustrate  trends  in  program  status 
throughout  various  stages  of  contract  completion.  These 
graphs  should  assist  program  analysts  in  providing  program 
managers  with  reasonable  contract  completion  cost  estimates 
for  contracts  in  various  categories  across  all  stages  of 
contract  completion. 
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AN  ANALYSIS  OF  ESTIMATE  AT  COMPLETION  MODELS  UTILIZING  THE 
DEFENSE  ACQUISITION  EXECUTIVE  SUMMARY  DATABASE 


I- _ introductioD 


Gflnfigal  laaug 

The  Reagan  Administration  was  elected  in  1980  with  a 
primary  goal  of  rebuilding  the  nation’s  defense 
capability.  A  significant  portion  of  this  rebuilding 
effort  included  the  development  and  production  of  needed 
major  weapons  systems  for  each  of  the  service  components 
of  the  Department  of  Defense  (DOD) . 

Though  similar  problems  in  the  weapons  procurement 
system  had  been  identified  in  post~World  War  II  systems 
development,  the  build-up  during  the  Reagan  years  brought 
increased  scrutiny  and  great  criticism  on  systems 
acquisition  and,  specifically,  on  cost  and  schedule 
overruns . 

This  heightened  enphasis  on  cost  and  schedule 
overruns  brought  concern  from  both  the  executive  and 
legislative  branches  of  government.  In  March  1981,  in  a 
memorandum  to  Defense  Secretary  Weinberger,  President 
Reagan  voiced  executive  direction; 

We  were  concerned,  as  I  am  sure  you  were,  to 
learn  of  the  significant  cost  growth  in  a  number 
of  Defense  programs  .  .  .  (10:33) 
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similar  thoughts  were  expressed  by  the  Chairman  of  the 
Defense  Appropriations  Subcommittee*  Rep  Joseph  P.  Addabbo 
(D-NY)  in  the  same  year: 

We  must  give  a  strong  message  to  the  Defense  * 

Department  that  the  old  way  of  doing  business  will 

not  be  tolerated.  .  .  .  Unless  we  demonstrate  we 

are  getting  a  handle  on  cost  overr\ins,  we'll  lose 

what  appears  to  be  a  consensus  to  rebuild  our 

military  forces.  (10:34) 

As  the  Reagan  Administration  progressed,  political 
pressure  mounted  regarding  the  management  of  major  weapons 
systems  programs.  In  1985,  Senators  Goldwater  and  Nunn, 
generally  viewed  as  hawks  on  defense  issues,  produced  a 
report  echoing  the  continuing  concern  of  Congress  over 
cost  and  schedule  increases.  This,  in  part,  led  to  a 
major  study  headed  by  former  Deputy  Defense  Secretary 
David  Packard,  known  as  the  President's  Blue  Ribbon 
Commission  on  Defense  Management.  Tnis  report  stated 
that : 


.  .  .  too  many  of  our  weapons  systems  cost  too 
much,  take  too -long  to  develop,  and  by  the  time 
they  are  fielded,  incorporate  obsolete 
technology.  (16:337) 


This  developing  consensus,  combined  with  federal 
deficit  pressures  and  declining  threat  assessments, 
seriously  damaged  the  political  will  present  in  1980  to 
develop,  produce  and  field  defense  weapons  systems. 
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The  A-12  program  cancellation  in  1991  graphically 
illustrated  that  cost  and  schedule  overrxins  would  no 
longer  be  met  with  additional  fxinding  and  the  acceptance 
of  slipped  development  and  production  time  frames. 

Background 

Though  cost  overruns  of  major  weapons  systems  gained 
heightened  attention  during  the  build-up  of  the  U.S. 
defense  budget  during  the  Reagan  Administration, 
historical  evidence  indicates  the  presence  of 
comparatively  higher  overruns  of  systems  produced  and 
developed  30  years  prior  (13:51).  A  study  by  Merton  Peck 
and  Frederick  Scherer  of  12  major  weapons  systems  produced 
in  the  1950 's  detailed  average  cost  growth,  from  start  to 
full-scale  development,  of  220  per  cent  (15:412).  A 
similar  RAND  Corporation  study,  published  in  1986,  notes  a 
range,  in  relation  to  current  programs,  of  approximately 
10  to  35  per  cent  (18:9). 

One  of  the  major  factors  in  the  decline  of  the  amount 
of  cost  overruns  was  the  introduction,  during  the  tenure 
of  Secretary  McNamara  at  the  Department  of  Defense,  of  a 
set  *)f  criteria  known  as  Cost /Schedule  Control  Systems 
Criteria  (C/SCSC) .  The  development  of  this  evaluation 
method  occurred  after  it  was  apparent  to  DOD  officials 
that  the  prior  methods  to  monitor  contract  performance 


were  inadequate  to  properly  gauge  major  weapons  systems 
programs . 

Prior  to  C/SCSC,  it  was  not  uncommon  for  contractors 
to  bid  low  on  prospective  contracts,  which  were  based 
primarily  on  fixed  fee  assus^tions,  and  then  petition  for 
alterations  to  the  contract  length  emd  price  during  the 
performance  phase  stating  that  they  had  not  realized  how 
conplex  and  costly  the  system  production  would  actually 
be.  The  government  was  often  iinaware  of  these  problems 
until  significantly  into  the  time  span  of  the  contract  and 
therefore  its  leverage  to  mandate  changes  to  solve  these 
problems  was  limited.  This  teoplate  for  disastrous  cost 
and  schedule  overrxms  necessitated  the  development  of  an 
evaluation  method  to  better  monitor  cost  and  schedule 
performance  (8:22). 

A  key  component  of  the  C/SCSC  system  is  the  Estimate 
at  Completion  (EAC)  calculation,  the  estimation  of  the 
total  program  cost  that  may  be  comput^ed  throughout  the 
program  life>cycle  based  upon  schedule  and  cost  data. 

This  figure  is  used  to  estimate  the  cost  variance  of  a 
program  from  its  original  baseline  and  is  an  essential 
driver  in  the  analysis  of  program  efficiency  and 
effectiveness . 

Even  with  the  relative  inqprovement  in  accounting 
measures  to  analyze  contractor  performance,  the  lack  of 
standardization  in  the  use  and  interpretation  of  EAC 


formulas  remains  a  problem.  Most  recently,  in  the 
Memorandum  for  the  Secretary  of  the  Naw.  Subi :  A-12 
Administrative  Ineruirv  (the  Beach  Report)  on  the  factors 
surrounding  the  cancellation  of  the  U.S.  Navy’s  A-12 
program,  cost  estimation  procedures  were  still  found 
woefully  lacking  due  in  part  to  disagreements  about 
selection  of  and  possible  manipulation  of  EAC  formulas 
(1:12-13) . 

As  a  remedy  to  potential  EAC  manipulation,  the  Beach 
Report  recommended  a  range  of  EACs  be  used  loi  future 
program  evaluations  (1:16).  A  range  of  EACs  will,  due  to 
the  relative  accuracy  of  the  various  models  based  upon 
contract  characteristics  and  stage  of  completion,  provide 
a  wider  'confidence  interval*  to  estimate  program  cost 
(9:3)  , 

Further,  the  Beach  Report  advised  changes  to  the  DOD 
5000  series  documents  indicating  the  Cost  Performance 
Index  (CPI)  should  be  used  as  a  benchmark  formula  for 
programs  over  15  per  cent  complete. 

(1)  Enter  the  range  of  estimates  at  completion, 
reflecting  best  and  worst  cases. 

(2)  Provide  the  estimate  at  completion  reflecting 
the  best  professional  judgment  of  the  servicing  cost 
analysis  organization.  If  the  contract  is  at  least 
15  per  cent  complete  and  the  estimate  is  lower  than 
that  calculated  using  the  cumulative  cost  performance 
index,  provide  an  explanation. 


(3)  Justify  the  program  manager’s  best 
estimate  (item  25)  if  the  contract  is  at  least 
15  per  cent  complete  and  the  estimate  is  lower 


than  that  calculated  using  the  cumulative  cost 
performance  index.  (7:16-K-6) 

The  CPI  was  chosen  as  a  baseline  EAC  due  to  evidence 
suggesting  its  relative  stability  as  a  predictor  of 
program  cost  from  the  20  per  cent  conpletion  point  across 
a  variety  of  program  and  contract  types  (4:7) . 

The  Schedule  Cost  Index  (SCI)  is  generally  viewed  to 
be  the  high-end  EAC  formula  (9:9) .  This  method  is 
theoretically  most  pessimistic  because  it  utilizes 
"equally*  both  schedule  and  cost  data  in  its  calculations 
with  potential  performance  negatives  in  each  measure 
aoqplifying  the  other,  generating  a  higher  total  EAC. 

Though  extensive  research  in  the  C/SCSC  emd  EAC  area 
has  been  conducted  for  approximately  20  years,  significant 
room  for  cost  estimation  process  refinement  and 
specification  remains.  Specifically,  while  the  CPI  euid 
SCI  have  been  established  in  practice  as  the  de  facto 
floor  and  ceiling,  respectively,  for  EAC  models,  research 
on  a  large,  diverse  program  database  has  not  definitely 
supported  these  contentions. 

Research  supporting  the  EAC  ceiling  and  floor 
assertion  would  be  most  useful  to  program  managers  and  the 
DOD  cost  community  as  it  would  allow  them  to  establish  eui 
area  in  which  the  Cost  at  Completion  (CAC)  would  most 
probably  be  contained.  Also,  it  would  confirm  existing 
suspicion  that  program  cost  estimates  outside  this  range 
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are  unrealistic  and  require  a  heightened  degree  of 
^cplauiation  and  justification. 

Finally,  the  trend  toward  declining  defense  budgets 
and  increasing  scrutiny  of  defense  acquisition  policy 
magnifies  the  importance  of  improving  the  management  and 
control  of  weapons  system  costs. 

t 

Research  Problem 

The  primary  objective  of  this  research  is  to  test  the 
IDOD  assertion  in  DOD  5002. 2 -M,  Defense  Acquisition 
Management  Documentation  and  Reports  and  Office  of  the 
Under  Secretary  of  Defense  for  Acquisition  {OUSD(A)) 
concept  that  the  Cost  Performance  Index  (CPI)*based  EAC  is 
a  valid  floor  and  the  Schedule  Cost  Index  (SCI) -based  EAC 
is  a  valid  ceiling  for  EAC  formulas  (7:16-H-6). 
Additionally,  this  research  explores  the  position  of  the 
Cost  at  Completion  relative  to  the  EAC  floor  and  ceiling. 

Specific  Problem  Statement 

Does  available  program  data  on  completed  DOD 
contracts  establish  a  range  of  EACs  consistent  with  DOD 
policy  and  assumptions?  The  primary  hypothesis  to  be 
tested; 


Hq;  Cost  at  coni)letion  is  bounded  below  by  the  CPI- 
based  EAC  and  above  by  the  SCI -based  EAC. 


The  hypothesis  was  answered  using  a  database 
described  below.  In  addition  to* the  CPI  and  SCI'based 
EACS/  the  SPI -based  EAC  was  selected,  for  a  total  of  three 
EAC  formulas. 

The  hypothesis  test  starts  with  determining  the 
percentage  deviations  of  each  of  four  EAC  formulas  from 
the  actual  CAC.  These  percent  deviations  from  CAC 
provided  a  normalized  set  of  data  points  for  comparative 
purposes.  The  percent  deviations  from  CAC  were  averaged 
and  the  EAC  with  the  highest  average  percent  deviation 
from  the  CAC  became  the  ceiling  and  the  EAC  with  the 
lowest  average  became  the  floor. 

The  l^othesis  was  valid  only  if  the  SCI-based  EAC 
was  the  ceiling,  the  CPI -based  EAC  was  the  floor,  and  if 
the  floor  and  ceiling  bounded  the  Cost  at  Completion. 

The  results  of  the  hypothesis  test  were  tested  for 
sensitivity  to  several  factors,  including  Index  Type, 
Contract  Completion  Stage,  Program  Phase,  Contract  Type, 
Branch  of  Service,  System  lype.  Major  Baseline  Changes  and 
Manage '  ^  Reserve . 

Tn«i  specific  categories  for  sensitivity  analysis  are 
addressed  with  investigative  questions,  following  a  brief 
description  of  the  database  used  in  this  analysis. 
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Degense  Aeauisibion  Exeeutive  Sinnmarv  Database 


OUSD(A)  provided  program  data,  compiled  from 
contractor  cost  management  reports  since  1972,  in  a 
database  known  as  the  Defense  Acquisition  Executive 
Summary  (DAES)  .  This  database  contains  performance  data 
on  over  500  major  weapons  system  programs  (major  defined 
as  research,  development,  test  and  evaluation  contracts 
over  $60  million  and  production  contracts  over  $250 
million)  from  each  of  the  service  branches  (19:5) . 

Investigative  Quest ions 

1.  Which  EAC  models  are  most  utilized  in  the  Department 
of  Defense  (DOD)  and  why? 

2.  Of  the  index-based  EACs  compared,  is  the  CPI-based  EAC 
the  floor  and  is  the  SCI -based  EAC  the  ceiling? 

3 .  Does  the  final  Cost  at  Con^letion  lie  within  the 
considered  range  of  EACs? 

4.  Which  EAC  is  the  most  accurate  predictor  of  final  Cost 
at  Completion? 

5.  Are  the  original  hypothesis  test  results  (answers  to 
questions  two  through  four)  sensitive  to: 

a.  Index  Type  (Cximulative,  Six  Month  or  Three  Month) 

b.  Program  Phase  (Pre-Production  or  Production) 

c.  Contract  Type  (Cost  Plus,  Firm  Fixed  or  Fixed 
Price) 

d.  Service  (Army,  Navy  or  USAF) 
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e.  System  Type  (Airframe,  Electronics,  Engine, 
Eguipment,  Ground,  Missile,  Ship  or  Space) 

f.  Major  Baseline  Changes  (Not  0TB  or  0TB) 

g.  Management  Reserve 

6.  Are  the  original  results  and  the  results  for  each 
category  sensitive  to  the  Contract  Cooq^letion  Stage? 

The  first  question  will  be  answered  in  the  literature 
review.  The  remaining  questions  will  be  answered  after 
analysis  of  the  DAES  database. 

SgopeyLimitat-ions 

The  primary  scope  of  this  thesis  is  utilization  of 
the  DAES  database  to  analyze  the  concept  that  CAC  is 
botmded  below  by  the  CPI -based  EAC  and  above  by  the  SCI- 
based  EAC.  Additional  analysis  focuses  on  the  sensitivity 
of  the  ceiling  and  floor  to  various  conditions  as 
described  in  the  investigative  questions. 

The  chief  limitation  of  this  thesis  is  the  quality 
and  consistency  of  the  DAES  database.  The  database 
exhibited  inconsistencies  in  terms  of  the  time  between 
consecutive  entries  of  program  cost  and  schedule  data.  A 
process  of  interpolation  within  the  initial  contract  data 
provided  the  information  necessary  to  minimize  this 
shortcoming. 

Having  provided  a  general  overview,  a  discussion  of 
the  specific  problem  statement  and  an  outline  of  the 
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investigative  questions,  it  is  useful  to  expand  on  the 
background  and  significance  of  this  research  with  a  review 
of  applicable  literature. 
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This  section  addresses  the  development  of  C/SCSC, 
defines  the  data  elements  used  in  EAC  calculations  and 
reviews  appropriate  and  applicable  EAC  comparison  studies. 

The  McNamara  Era  in  the  DOD  saw  the  introduction  of 
many  concepts  used  in  the  civilian  sector  to  the  management 
of  national  defense.  Two  mechanisms  introduced  prior  to 
the  development  of  C/SCSC  were  attempts  to  solve  the  same 
problems;  but  met  with  limited  success.  Program  Evaluation 
euxd  Review  Technique  (PERT)  and  the  Critical  Path  Method 
(CPM)  were  industry  and  academic  concepts  that  were 
essentially  techniques  to  monitor  and  optimize  the 
scheduling  side  of  the  equation.  Though  these  techniques 
were  a  step  in  the  right  direction  regarding  schedule 
monitoring,  they  did  not  initially  take  into  account  the 
cost  element;  and  additionally  were  fought  by  members  of 
both  the  military  and  contractor  communities  due  to  the 
implementation  style  of  McNamara's  'whiz  kids*  (8:23).  A 
by-product  of  the  introduction  of  these  concepts  was  the 
initiation  of  DOD  to  the  'earned  value*  theory,  the 
conceptual  linchpin  of  C/SCSC.  This  method  of  analyzing 
contract  performance: 

.  .  .  suggested  the  idea  of  planning  a  program 
and  the  necessary  resources  in  sufficient  detail 
so  as  to  allow  for  the  precise  measurement  of 
performance  along  the  way,  and  of  having  the 
ability  to  obtain  reliable  estimates  of  the  total 
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costs,  and  total  tiroes  needed  to  coroplete  the 
various  programs .  (8:23) 


The  genesis  of  C/SCSC  was  the  formation  of  a  United 
States  Air  Force  body  known  as  the  Cost /Schedule  Planning 
and  Control  Specification  Group.  This  group  was  tasked  to 
develop  criteria  that  would  not  replace  the  internal 
management  control  systems  of  defense  contractors  but 
would  rather  provide  the  government  with  a  means  to 
evaluate  a  contractor's  present  systems  in  relation  to 
cost  control  and  schedule  performance  (8:24). 

In  late  1967,  the  Air  Force  study  group's  efforts 
resulted  in  the  publication  of  DOD  Instruction  7000.2, 
Performance  Measurement  for  Selected  Acouisitions .  This 

document  introduced  35  criteria  that  are  partitioned  into 
5  major  areas: 

Organization — To  define  the  contractual  effort 
with  use  of  a  work  breakdown  structure,  assign 
responsibilities  for  performance  of  the  work,  and 
accoK^lish  all  this  with  use  of  an  integrated 
contractor  management  control  system. 

Planning  and  Budgeting — To  establish  and  maintain 
a  performance  measurement  baseline  for  control  of 
the  work. 

Accounting — To  accumulate  the  costs  of  work  and 
materials  in  a  manner  which  allows  for  comparison 
with  earned  value. 

Analysis — To  measure  earned  value,  to  analyze 
variances  of  both  costs  and  schedules,  and  develop 
reliable  estimates  of  costs  at  completion. 

Revisions  and  Access  to  Data — To  incorporate 
changes  to  the  controlled  baseline  as  required, 
and  allow  appropriate  Government  representatives 
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to  have  access  to  contract  data  for  determining 
contract  compliance.  (8:26*27) 

The  analysis  criteria  ensure  that  data  collected  and 
maintained  by  the  contractor  include  sufficient 
information  to  allow  the  government  to  analyze  contractor 
performance.  Program  managers: 

.  .  .  require  a  comparison  of  actual  vs.  planned 
performance,  calculation  of  variances,  and  analysis 
of  variances  (if  they  exceed  predetermined 
thresholds).  (11:13-18) 

The  measures  or  ratios  that  the  government  uses  to  deduce 
the  quality  of  contractor  performance  include  Budgeted 
Coat  of  Worlc  Scheduled  (BCWS) ,  Budgeted  Cost  of  Wor]c 
Performed  (BCWP) ,  Actual  Cost  of  Wor)c  Performed  (ACWP) , 
Budget  at  Completion  (BAC)  and  Estimate  at  Completion 
(EAC)  as  defined  in  Appendix  A.  Additional  useful 
measures  include  Management  Reserve  (MR) ,  Total  Allocated 
Budget  (TAB) ,  Percent  Complete  (PC)  and  Cost  at  Completion 
(CAC),  also  defined  in  ^pendix  A. 

The  basic  EAC  formula  used  in  this  analysis  is  index- 
based. 


EAC  =  ACWP  +  (TAB  -  BCWP) /index 


(1) 
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Index  DaCinitiQng 

1.  Cost  Perfonnance  Index  (CPI).  The  CPI  is  obtained  by 
dividing  BCWP  by  ACWP.  A  ratio  greater  than  one 
(BCWP>ACWP)  indicates  a  cost  underrun.  A  ratio  less  than 
one  (BCWP<ACWP)  indicates  a  cost  overrun.  The  three  CPIs 
analyzed  include  cuxnulative,  six-month  and  three-month,  as 
defined  below. 

CPIcum  a  BCWPcxun/ACWPcum  (2) 

-5  -5 

CPI6  =  I  BCWP/  I  ACWP  (3) 

iaO  isO 

where  i  represents  the  month  and  goes  from  current  (iaO) 
to  five  months  prior  (iaS)  to  provide  the  six  most  recent 
data  points. 


-2  -2 

CPI3  a  I  BCWP/I  ACWP  (4) 

i*0  iaO 

2.  Schedule  Perfonnance  Index  (SPI) .  The  SPI  is  the 
ratio  of  BCWP  to  BCWS.  A  ratio  greater  than  one 
(BCWP>BCWS)  indicates  a  program  is  ahead  of  schedule  while 
a  ratio  less  than  one  (BCWP<BCWS)  indicates  a  program  is 
behind  schedule.  The  cumulative,  six  and  three  month  SPIs 
are  defined  below. 

SPIcimn  =  BCWPcum/ BCWS  cum  (5) 
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(6) 


-5  -5 

SPI6  -  2  BCWP/I  BCWS 
iaO  isQ 

-2  >2 

SPI3  »  2  BCWP/2  BCWS  (7) 

iaO  i=0 

3.  Schedule  Cost  Index  (SCI).  The  SCI  is  the  product  of 
CPI  and  SPI.  Cumulative,  six  and  three  month  indexes  are 
defined  below. 

SCIcxim  a  CPIcum  *  SPIcum  (8) 

SCI6  =  CPIS  *  SPI6  (9) 

SCI3  =  CPI3  *  CPIS  (10) 


Having  ac]cnowledged  the  index  definitions,  additional 
clarification  of  the  relationship  between  an  index  and  its 
corresponding  EAC  is  best  illustrated  with  an  example. 

The  following  exanple  meets  the  conditions  of  the 
hypothesis.  The  relative  ranking  of  EACs  results  in  the 
CAC  boxanded  by  the  SCI -based  EAC  as  the  ceiling  and  the 
CPI -based  EAC  as  the  floor. 


A  typical  contract  might  be  expected  to  be  behind 
schedule  and  over  cost.  This  condition  is  exemplified 
with  CPI  and  SPI  values  both  less  than  one. 
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Let  TAB-100  BCWP-27  BCW3-33.75  ACWP=30  CAC=125 

INDEX  CALCULATIONS :  CPI=BCWP/ACWP» .  9 

SPI=BCWP/BCWSs.8 

SCI-CPI*SPI-.72 

EAC  CALCULATIONS;  EAC  =  ( (TAB-BCWP) /index) +ACWP 

«ACcpi=< (100-27) /.9)+30=lll. 11 
EACsPi=( (100-27)/. 8)+30=121. 25 
EAC*cx=( (100-27) / .72)+30=131.9 

When  CPI  end  SPI  are  less  than  one,  their  product 
(SCI)  will  be  less  than  either  CPI  or  SPI.  This  example 
illustrates  how  the  lowest  index  (SCI)  has  the  highest 
corresponding  EAC,  since  the  index  falls  in  the 
denominator  of  the  basic  SAC  formula. 

Understanding  the  mechanics  of  the  indexes,  the  EACs, 
emd  how  they  relate  to  each  other  paves  the  way  for 
further  exploration  into  the  significance  of  EACs  through 
a  review  of  EAC  literature  to  date.  A  review  of  several 
comprehensive  EAC  studies  follows  to  help  the  reader 
appreciate  the  variety  of  conclusions  to  date.  These 
studies  are  summarized  in  Table  1. 
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TABLE  1 


SUMMARY  OF  EAC  RESEARCH 


Somce 

Tonic 

Twp^nwilwrnf 

Contracts 

ConciitthiM 

Bright  &  Howard 
(1981) 

Army 

EAC  Formulas 

11  developmeDt 

EAC  selection  should  be 
based  upon  program 
characiBristics;  SG6 
mggested  optimal 

(Kovach,  Haydoo  ft 
Reiiher(1982) 

Air  Ferae 

EAC  Formulas 

15  develoinent 

Cn  vatiadoos  most 
accurate 

Reidel  AOttnce 
(1989) 

Air  Ferae 

EACFotinulas 

Iddevelopnem 

40prodai:^ 

No  single  EAC  method 
ptoduca  optimal 
esdmaies  for  all  types  of 
programs 

Beach  (1990) 

Navy 

Caacellsrioc 

1  development 

EAC  cakniladoos  vary 
greatly  ft  are  easily 
manipulated 

McKinney  (1990) 

Air  Force 

EAC  Formulas 

N/A 

Provided  history  and 
review  of  EAC  research 

Cbristenaen, 
AntoUnlft 
McKinney  (1992) 

Air  Force 

EAC  Formulas 

N/A 

Accuracy  of  regressicn- 
based  formulas  over 
index>ba8ed  not 
supported 

Fleming  (1992) 

Civilian 

Coonactor 

EAC  Formulas 

N/A 

CFIcum  is  tnkhile  range 
formula;  S(^Icam  ishigb 
end  formula 

Christensen  (1993) 

Air  Force 

EAC  Formulas 

N/A 

EAC  formula  acctaacy 
depends  upon  system 
ty^,  phase  aud  type  of 
contract 

Christensen  ft 
Heise(1993) 

Air  Force 

CPlSiabiUcy 

62  development 
93  produoioo 

CPIcum  is  stable  frum 
the  20%  compledon 

Doim 
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BAC  Conpariaon  smdieg 


The  EAC  forac&sts  the  completed  cost  of  a  defense 
contract  and  provides  a  basis  to  systematically  project 
total  weapons  system  cost,  a  necessary*  and  key  conqponent 
of  the  governmental  oversight  process  lacking  in  the  era 
prior  to  C/SCSC.  Calculation  and  emalysis  of  EAC  data 
provides  the  ability  to  determine  the  relationship  of  cost 
and  schedule  parameters  throughout  the  program  life-cycle, 
allowing  the  government  a  mechanism  to  identify  and 
minimize  cost  and  schedule  ovennins. 

Numerous  atten^ts  have  been  made  to  compare  and 
contrast  the  performance  of  various  EAC  formulas  based 
upon  what  invariably  have  been  rather  small  and 
homogeneous  weapons  systems  databases. 

In  1980  and  1981,  the  ManTech  International 
Corporation,  in  a  study  contracted  by  the  U.S.  Navy, 
tested  24  EAC  formulas  on  a  database  of  15  development  and 
6  production  contracts.  Twelve  of  these  formulas  ware 
index-based  with  the  remainder  regression-based  (S:l-7, 
62-65)  .  Also  in  1981,  Bright  emd  Howard  tested  11  EAC 
formulas  on  11  U.S.  Army  development  contracts.  Nine  of 
the  formulas  were  index-based  with  the  remainder 
regrsssicn-based.  These  studies  concluded  that 
regression-based  formulas  tend  to  perform  well  in  the 
early  stages  of  conpleticn.  The  index  formulas  that 
performed  the  best  across  all  three  stages  were  CPI -based 


They  also  recoxnmended  the  SCI -based  EAC  as  a  candidate  for 
the  most  accurate  predictor  of  Cost  at  Conpletion  (3:217- 
221)  . 

In  1989,  Reidel  and  Chance  tested  6  index -based 
formulas  on  a  U.S.  Air  Force  database  consisting  of  16 
development  and  40  production  contracts.  These  programs 
were  evaluated  at  four  finite  stages  of  conpletion  versus 
the  early,  middle  emd  late  range  methods  of  the  ManTech 
and  Bright  and  Howard  studies.  Their  results  indicated 
that  SCI  and  CPI-based  models  performed  well  across  all 
stages  of  completion  (17:3-6,  72-78). 

Regarding  empirical  support  of  the  CPI -floor  and  SCI- 
ceiling  assertions,  the  available  studies  are  suggestive 
but  not  conclusive.  Bright  and  Howard  found  that  CPIcum 
does  tend  to  be  the  floor  among  the  various  CPI  formulas 
(2:16)  . 


.  .  .  on  the  average,  the  (EAC)  technique  based  upon 
the  most  recent  data  dominates  (exceeds)  over  the 
technique  based  upon  less  recent  data  (2:16) . 


Fleming  suggests  the  SCIcum  as  a  ceiling  among  commonly 
used  EAC  formulas  included  in  popular  EAC  software 
pac]cages . 


.  .  .  We  at  Micro-Frame  consider  this  (CPI*SPI)  to  be 
a  high  end  formula  .  .  .  (9:9) 
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Finally,  the  Beach  Report  provides  anecdotal  support  for 
the  reasonableness  of  the  CPlcum  floor  and  SCicuin  ceiling 
based  upon  the  cancelled  A*12  program. 

These  representative  studies  illustrate  the  general 
conclusions  and  weaknesses  of  the  EAC  reb'earch  conducted 

over  the  past  20  years.  While  some  formulas  tend  to  be 

/ 

preferred,  McKinney  reviewed  EAC  research  in  1992  and 
found  a  lack  of  consensus  remains  regarding  the 
appropriateness  of  the  available  formulas  and  formula 
types  (14:76-79) . 

Recall  from  Chapter  One  that  the  first  investigative 
question  asked  the  following: 

1.  Which  EAC  is  the  most  accurate  predictor  of  final  Cost 
at  Conqpletion? 

As  indicated  in  the  review  of  EAC  research  to  date, 
there  is  no  one  EAC  singled  out  as  the  most  accurate.  ne 
CPI/  SPI  and  SCI -based  EACs  are  most  often  used  to 
estimate  program  costs  but  there  is  no  clear  pattern  of 
conditions  under  which  each  index  is  most  appropriate. 

The  small  and  homogeneous  databases  upon  which  the 
conparisons  are  based  present  a  significant  shortcoming. 
The  small  san^le  sizes  reduce  the  certainty  of  statistical 
conclusions  and  inhibit  the  ability  to  generalize  results . 
What  is  missing  is  a  comprehensive  test  of  EAC  formulas  on 
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a  large,  heterogeneous  database  to  better  ascertain  the 
validity  of  EAC  formulas. 

Our  research  is  designed  to  address  this  limitation 
by  performing  analysis  of  EAC  models  based  upon  programs 
in  the  DAES  database.  This  database  provides  larger 
san^Ie  sizes  from  a  more  con^lete  and  diverse  set  of 
program  data  than  previously  available  to  those 
undertaking  EAC  research. 

The  methodology  for  testing  the  hypothesis  and 
answering  the  remaining  investigative  questions  is 
presented  in  the  following  chapter. 
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III. _ Methodology 


This  chapter  describes  the  procedures  used  to  test 
the  hypothesis. 

Hq:  Cost  at  Con^iletion  is  bounded  below  by  the 
CPI -based  EAC  and  above  by  the  SCI -based 
EAC. 

Introduction 

Before  discussing  the  methodology  applied  to  test  the 
hypothesis,  the  database  itself  merits  attention.  A 
description  of  the  database  and  its  fields  is  followed  by 
an  explanation  of  the  process  used  to  prepare  the  database 
for  use  (referred  to  as  data  definition) .  The  data 
description  and  data  definition  are  followed  by  an 
explanation  and  justification  of  the  approach  used  to  test 
the  hypothesis  and  an  explanation  of  the  actual  EAC 
calculations  and  graph  preparations  for  data  analysis. 

Ihe  Database 

A  sample  of  321  contracts  from  the  DAES  database  was 
selected  for  analysis  to  test  the  hypothesis.  The 
original  database  contains  approximately  500  contracts  and 
16,000  quarterly  report  dates  of  information.  The 
database  has  various  Department  of  Defense  contracts  which 
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date  from  November,  1972  to  October,  1992.  The  fields  of 
interest  for  testing  the  hypothesis  include  descriptive 
and  numeric  fields  as  described  below. 

DeacriptiYe..Fieldg 

1.  Project  Number  (PNO) .  This  identifies  a  group  of 
contracts  that  belong  to  a  particular  weapon  system 
program. 

2.  Contract  Nxjmber  (CNO)  .  This,  in  conjunction  with  its 
corresponding  PNO,  uniquely  identifies  each  of  the  321 
contracts  used  in  the  DAES  database. 

3.  Submit  Date.  This,  in  conjunction  with  its 
corresponding  PNO  and  CNO,  imiquely  identifies  each  of  the 
3803  records  used  in  the  database.  The  submit  date  is 
'nportant  for  interpolating  missing  data. 

4.  Phase.  This  identifies  the  phase  as  either  Pre* 
Production  (PREP),  Production  (PROD)  or  Other  (OTHER). 

5.  Contract  lype  (CTYPE) .  This  identifies  the  type  of 
contract  as  Cost  Plus  (CP),  Firm  Fixed  (FF)  or  Fixed  Price 
(FP)  . 

6.  Branch  of  Service  (SERVICE).  This  identifies  the 
branch  of  service  initiating  the  contract:  Air  Force  (F) , 
Army  (A) ,  or  Navy  (N) . 

7.  System  Type.  This  identifies  the  contract  by  the  type 
of  system:  Airframe  (A),  Electronics  (E) ,  Engine  (N)  , 
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Equipment  (Q) ,  Ground  (G) ,  Missile  (M) ,  Ship  (S),  Space 
(P) ,  or  Other  (OTHER) . 

8.  Over  Target  Baseline  Date  (0TB  DATE).  This  field 
identifies  contracts  without  an  0TB  date  (S)  and  contracts 
whose  budget  baselines  went  Over  Target  (in  which  case  the 
appropriate  date  is  recorded  instead  of  an  *S") . 

Table  2  displays  contract  categories,  their  sample 
sizes  and  basic  descriptive  statistics  (mean  and  standard 
deviation)  of  the  Cost  at  Completion  and  Total  Allocated 
Budget  for  each  category. 

Mumeric  -fialdg 

The  numeric  fields  of  interest  in  the  database 
include  ACWP,  BCWP,  BCWS,  TAB  and  MR,  as  defined  in 
Appendix  A.  Other  numeric  fields  of  interest,  as  well  as 
the  index  and  EAC  calculations,  are  all  computed  from 
these  basic  fields. 


25 


TABLE  2 


CONTRACT  COMPOSITION  BY  CATEGORY 


Category 

R^SEUil 

msm 

BSSQSSl 

All  Contracts 

321 

3803 

(316,466) 

(292,466) 

Phase 

PrC'Production 

112 

1454 

(385,596) 

(347,570) 

Production 

135 

1398 

(304,418) 

(28434) 

Others 

74 

951 

Cost  Plus 

137 

709 

(297,514) 

(276,499) 

Firm  Fixed 

13 

96 

(215,127) 

(220,127) 

Fixed  Price 

169 

2053 

(337.434) 

(310,410) 

Other* 

2 

945 

Air  Force 

127 

1638 

(331,365) 

(306.345) 

Army 

48 

606 

(214,276) 

(173.236) 

Navy 

144 

1524 

(346,601) 

(331,582) 

Other* 

2 

35 

System  Type 

Airframe 

22 

228 

(792,732) 

(753,684) 

Electronics 

43 

540 

(227,234) 

(212,217) 

Engine 

12 

117 

(183,175) 

(184,177) 

Equipment 

19 

212 

(205,259) 

(195,230) 

Ground 

8 

86 

(74,70) 

(54,50) 

Missile 

43 

515 

(476,763) 

(446,746) 

Ship 

19 

244 

(610,719) 

(567,692) 

Space 

25 

339 

(287,285) 

(219,214) 

Other* 

130 

1522 

1  IN  III! 

Stable 

253 

3551 

(319,470) 

(296,451) 

Unstable 

68 

252 

(274,404) 

(246,369) 

1  CAC  refers  lo  Cost  at  Completioa.  This  is  tbe  final  ACWP  in  each  cootracL 

2  TAB  refers  to  Total  Allocated  Budget 

3  Other  refers  to  oootraca  that  did  not  cleariy  &il  into  the  appropriate  sensitivity 
categories.  They  were  therefore  excluded  from  these  caiegori^  analyses  and 
summary  statistics  for  '’Other*  were  not  pertinent 
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Pflttt  Definition 


One  of  the  first  steps  involved  querying  the  original 
database  for  records  with  zero  values  to  delete  these  from 
consideration.  There  also  were  several  contracts  with 
fewer  than  five  report  dates  of  information  which  were 
removed  from  consideration.  Finally,  those  contracts 
which  did  not  contain  data  from  20  to  80  percent  complete 
were  removed  from  the  database.  These  initial  criteria 
eliminated  contracts  which  lacked  data  characteristics 
necessary  to  test  the  hypothesis.  This  resulted  in  a 
usable  database  of  321  contracts  with  varying  numbers  of 
report  dates,  for  a  total  of  3803  DAES  reports  of 
information.  Once  the  usable  data  was  extracted,  it  was 
necessary  to  identify  logical  categories  for  comparison  in 
the  analysis. 

Defining  descriptive  field  entries  required  queries 
that  searched  for  a  particular  entry  or  set  of  entries  a.  i 
changed  those  entries  to  a  desired  character.  The  field 
identifying  Service  needed  no  adjustments.  The  Contract 
Types  had  approximately  25  different  versions  which  were 
all  defined  as  one  of  four  categories  (CP,  FF,  FP,  or 
OTHER) .  Phases  were  all  defined  as  Pre-Production  or 
Production.  System  lypes  were  derived  from  a  field  called 
Contract  Description  (CDES) .  From  a  list  of  several 
hundred  descriptions,  each  entry  was  assigned  one  of  the 
eight  categories  identified  above  (or  OTHER,  where  the 
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system  fit  none  of  the  eight  identified  categories) . 
Finally,  0TB  was  adjusted  to  contain  an  "S*  for  contracts 
which  did  not  contain  0TB  dates.  Upon  completion  of  data 
definition,  it  was  necessary  to  address  the  problem  of 
missing  data  through  a  process  of  interpolation.  The 
interpolation  process  is  described  in  detail  in  Appendix 
B. 

Having  described  the  database  composition  and  the 
process  used  to  prepare  the  database  for  analysis,  the 
focus  turns  to  an  overview,  an  explanation  and  a 
justification  of  the  approach  to  test  the  hypothesis. 

Oyerview  o£  Hypothesia  Teat 

There  were  three  EAC  values  chosen  to  test  the 
hypothesis.  These  included  the  CPI-based  EAC,  the  SPI- 
based  EAC  and  the  SCI-based  EAC,  all  of  which  are  defined 
in  Chapter  Two.  These  three  EAC  formulas  were  normalized 
into  percent  deviation  from  CAC.  The  EAC  with  the  lowest 
average  percent  deviation  from  CAC  was  defined  as  the 
floor  while  the  EAC  with  the  highest  average  percent 
deviation  from  CAC  was  defined  as  the  ceiling. 

The  range  of  ceiling  and  floor  was  evaluated  to 
determine  if  the  CAC  was  bounded  by  them.  If  zero  average 
percent  deviation  from  CAC  fell  within  the  range  of  the 
ceiling  and  floor,  then  it  was  logical  to  conclude  that, 
on  average,  CAC  was  bounded  by  the  ceiling  and  floor.  The 
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EAC  whose  average  percent  deviation  from  the  CAC  was 
closest  to  zero  was  defined  as  the  most  accurate  predictor 
of  the  CAC. 


Explanation  of  Approaeh 

The  hypotliesis  was  tested  with  descriptive  statistics 
on  the  percent  deviations  from  CAC  of  the  three  EAC 
formulas.  The  mean  was  the  key  statistic  used.  The 
standard  deviation  (SD)  and  coefficient  of  variation  (CV) , 
where  CV  is  the  percentage  represented  by  the  ratio  of  SD 
to  the  mean,  were  also  calculated  to  provide  additional 
information  about  the  data  dispersion.  The  Coefficient  of 
Variation  was  calculated  mainly  because  it  provided  a 
measure  of  relative  dispersion  across  different  EAC 
formulas. 


.  .  .  The  standard  deviation  cannot  be  the  sole  basis 
for  comparing  two  distributions.  If  we  have  a 
standard  deviation  of  10  and  a  mean  of  5,  the  values 
vary  by  an  amount  twice  as  large  as  the  mean  itself. 
If,  on  the  other  hand,  we  have  a  standard  deviation 
of  10  and  a  mean  of  5,000,  the  variation  relative  to 
the  mean  is  insignificant.  Therefore,  we  cannot 
know  the  dispersion  of  a  set  of  data  until  we 
know  the  standard  deviation,  the  mean,  and  how  the 
standard  deviation  compares  to  the  mean.  The 
coefficient  of  variation  ( (SD/Mean) *100)  is  one 
such  relative  measure  of  dispersion  .  .  .  (12;  130) 


If  the  CVS  differ  significantly  among  EAC  formulas,  this 
would  indicate  that  some  EACs  have  wide  data  dispersion 
while  others  have  relatively  narrower  data  dispersion.  If 
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two  EACs  have  equal  means,  then  the  EAC  with  the  smaller 
CV  (narrower  data  dispersion)  would  probably  be  a  more 
reliable  predictor  of  CAC  than  the  EAC  with  the  larger 
EAC. 


Justification  for  Approach 

Due  to  data  dependence,  both  across  EAC  values  and 
down  through  report  dates,  it  was  inappropriate  to  do  a  t- 
test  on  the  difference  between  EAC  means  to  confirm 
statistical  differences  among  che  means.  However,  the 
mean  provides  the  best  available  measure  of  central 
tendency  to  compare  the  EAC  formulas. 

Tea  ting -the  HYPO-thesig 

The  three  EACs  evaluated  were  based  on  the  CPI,  SPI 
and  SCI .  Three  Index  Types  for  each  of  these  indexes  were 
evaluated  (cumulative,  six-month  and  three-month)  for  a 
total  of  nine  EAC  formulas. 

Indeat.and  EAC.Calcuiatigna 

Each  of  the  nine  indexes  and  EACs  was  calculated  for 
each  of  the  3803  DAES  report  dates.  The  cumulative 
calculations  used  the  actual  report  data  values . 

The  three-month  indexes  used  the  three  most  recent 
months  of  report  data. 
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Exanmle  Calculation  of  Ttiree-Month  Indexes 
CPI3  =  (BCWP+BCWP1+BCWP2)  /  (ACWP+ACWP1+ACWP2) 

The  six-month  indexes  used  the  six  most  recent  months 
o£  data. 

Example  Calculation  of  Six-Month  Indexes 

CPI6  =  {BCWP1-BCWP1+  .  .  .BaVPS)  /  (ACWP+ACWP1+  .  .  .+ACWP5) 

The  EACs  were  calculated  using  their  corresponding 
indexes.  The  nine  EACs  are  defined  as  follows. 


EACl 

s 

ACWP 

+ 

(TAB 

- 

BCWP) /CPIcum 

(14) 

EAC2 

» 

ACWP 

+ 

(TAB 

- 

BCWP)/CPI6 

(15) 

EAC3 

= 

ACWP 

(TAB 

- 

BCWP)/CPI3 

(16) 

EAC4 

s 

ACWP 

* 

(TAB 

- 

BCWP)/SPIcum 

(17) 

EACS 

= 

ACWP 

+ 

(TAB 

- 

BCWP)/SPI6 

(18) 

EAC  6 

s 

ACWP 

+ 

(TAB 

- 

BCWP) /SPI3 

(19) 

EAC7 

= 

ACWP 

+ 

(TAB 

- 

BCWP) /SCIcum 

(20) 

EACS 

= 

ACWP 

+ 

(TAB 

BCWP) /SCI6 

(21) 

EAC9 

s 

ACWP 

+ 

(TAB 

- 

BCWP) /SCI3 

(22) 

Upon  calculating  EAC  values,  the  next  step  toward 
testing  the  hypothesis  was  to  normalize  the  data  for 
comparison  purposes. 


Momalizinq  EAC  Valusg 


As  defined  in  Appendix  A,  Cost  at  Contpletion  (CAC) 
was  defined  to  be'  the  final  ACWP  value  in  a  contract.  CAC 
is  necessary  to  normalize  the  EAC  data  into  percent 
deviation  from  final  cost.  Failure  to  normalize  the  data 
would  cause  larger  contracts  to  overshadow  the  smaller 
ones.  Percent  deviation  from  CAC  puts  all  contracts  on 
level  groimd  for  comparative  purposes  and  is  calculated  as 
follows . 

Percent  Deviation  from  CAC  =  ((EAC  -  CAC) /CAC)  *  100  (23) 

These  percent  deviations  were  analyzed  in  testing  the 
hypothesis.  With  the  data  normalized,  the  descriptive 
statistics  were  the  next  calculations  of  interest. 

Pftacriptiyft.  atar.iaLicg 

The  mean,  standard  deviation  and  coefficient  of 
variation  for  all  nine  EACs  were  calculated  for  Overall 
and  for  three  Contract  Con5>letion  Stages,  defined  by 
Percent  Complete  (PC)  as  follows: 

Early;  PC<e.35 
Middle:  .35<PC<*.70 


Late: 


PC>.70 


The  cutoff  points  of  35  and  70  percent  were  chosen  to 
divide  the  data  roughly  into  thirds  to  allow  for 
generalizations  based  upon  the  Contract  Completion  Stage. 

These  were  made  for  five  categories  (Phase,  Contract 
Type,  Service,  System  Type  and  Major  Baseline  Changes) . 

The  means  were  compared  to  decermxne  the  floor  (minimxjm 
average)  and  the  ceiling  (maximum  average)  for  each  stage 
of  conpletion  within  each  category  and  overall.  In 
addition  to  determining  the  ceiling  and  floor,  it  was 
necessary  to  determine  if  the  CAC  fell  within  the  ceiling 
and  floor. 

The  CAC  was  bounded  by  the  ceiling  and  floor  only  if 
the  ceiling  had  an  average  percent  deviation  from  the  CAC 
that  was  greater  than  ze'o  and  the  floor  had  an  average 
percent  deviation  from  the  CAC  that  was  less  than  zero. 
(Since  an  average  deviation  from  the  CAC  of  zero 
represents  the  CAC  itself,  it  can  only  be  bounded  by  the 
ceiling  and  floor  if  zero  falls  between  the  ceiling  and 
floor.)  This  general  approach  applied  to  answering  the 
remaining  investigative  questions. 

The  following  portion  of  this  methodology  explains 
the  procedures  used  to  answer  investigative  questions  two 
through  six.  In  addition  to  explaining  how  these 
questions  are  answered,  a  brief  description  is  provided  of 
how  graphs  were  prepared  to  provide  additional  information 
for  analysis  of  trends  in  the  data. 
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2.  Of  the  index-based  EACs  compared,  is  the  CPI-based  EAC 
the  floor  and  is  the  SCI-based  EAC  the  ceiling? 

This  question  was  answered  using  two  procedures.  The 
first  was  to  calculate  the  average  percent  deviation  from 
the  Cost  at  Conpletion  (CAC)  for  each  EAC.  This  step 
normalized  the  data  to  prevent  more  costly  contracts  from 
shadowing  the  effects  of  less  costly  contracts.  These 
average  percent  deviations  from  the  CAC  were  compared  to 
determine  the  maximum  and  the  minimum  average  percent 
deviation  from  the  CAC.  The  minimum  value  was  the  floor 
and  the  maximum  value  was  the  ceiling. 

3 .  Does  the  final  Cost  at  Conviction  lie  within  the 
considered  range  of  EACs? 

This  question  was  answered  by  comparing  the  minimum 
average  percent  deviation  from  the  CAC  to  the  maximum 
average  percent  deviation  from  the  CAC.  The  CAC  was 
within  the  range  of  EACs  only  if  the  minimum  average 
percent  deviation  from  the  CAC  was  less  than  zero  and  the 
maximum  average  percent  deviation  from  the  CAC  was  greater 
than  zero.  As  long  as  the  EAC  floor  was  less  than  the  CAC 
and  the  EAC  ceiling  was  greater  than  the  CAC,  the  CAC  was 
within  the  range  of  EACs . 
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4.  Which  EAC  is  the  most  accurate  predictor  of  final  Cost 
at  Completion? 

The  most  accurate  predictor  of  final  Cost  at 
Completion  was  defined  as  the  EAC  closest  to  the  CAC.  In 
the  case  where  both  the  ceiling  and  floor  underestimated 
the  CAC,  the  most  accurate  predictor  of  the  CAC  was  the 
ceiling.  In  the  case  where  both  the  ceiling  and  floor 
overestimated  the  CAC,  the  floor  was  the  most  accurate 
predictor  of  the  CAC.  When  the  CAC  was  bounded  by  the 
ceiling  and  fi.oor,  it  was  possible  for  any  of  the  three 
EACs  to  be  the  most  accurate  predictor  of  the  CAC. 

5.  Are  the  original  hypothesis  test  results  (answers  to 
questions  two  through  four)  sensitive  to: 

a.  Index  Type  (Cumulative,  Six-Month  or  Three-Month) 

This  question  was  answered  by  comparing  the  three 
cumulative  index-based  EACs  (EACl,  EAC4  and  EAC7),  the 
three  six-month  index-based  EACs  (EAC2,  EACS  and  EACS)  and 
the  three  three-month  index-based  EACs  (EAC3,  EAC6  and 
EAC9) .  In  all  three  cases,  if  the  maximum  and  minimum 
average  percent  deviation  from  the  CAC  remained  the  same, 
if  the  position  of  the  CAC  relative  to  the  floor  and 
ceiling  remained  the  same,  and  if  the  same  EAC  was  the 
most  accurate  predictor  of  the  CAC,  then  the  original 
results  were  not  sensitive  to  the  Index  Type. 
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b.  Program  Phase  (Pre-Production  or  Production) 

This  question  was  answered  by  determining  the  EAC 

minimum  and  maximum  average  percent  deviations  from  the 
CAC  for  both  the  Pre-Production  and  the  Production 
contracts.  If  the  results  were  the  same  as  the  original 
results,  if  the  position  of  the  CAC  relative  to  the  floor 
and  ceiling  remained  the  same,  and  if  the  same  EAC  was  the 
most  accurate  predictor  of  the  CAC,  then  the  results  were 
not  sensitive  to  Program  Phase. 

c.  Contract  Type  (Cost  Plus,  Firm  Fixed  or  Fixed 
Price) 

This  question  was  answered  by  determining  the  minimum 
and  the  maximxim  average  percent  deviation  from  the  CAC  for 
each  Contract  lype  (Cost  Plus,  Firm  Fixed  and  Fixed 
Price) .  There  were  some  contracts  in  the  database  which 
contained  combinations  of  these  Contract  Types .  Those 
contracts  which  contained  more  than  one  Contract  Type  and 
those  contracts  whose  Contract  Type  could  not  be 
identified  were  not  included  in  this  sensitivity  analysis. 
If  the  results  agreed  with  the  original  results,  if  the 
position  of  the  CAC  relative  to  the  ceiling  and  floor 
remained  the  seune,  and  if  the  same  EAC  was  the  most 
accurate  predictor  of  the  CAC,  then  the  results  were  not 
sensitive  to  Contract  Type. 
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d.  Service  (Artry,  Navy  or  USAS') 

This  sensitivity  was  tested  by  determining  the 
minimum  and  maximum  average  percent  deviation  from  the  CAC 
for  each  Branch  of  Service.  If  the  results  agreed  with 
the  original  results,  if  the  position  of  the  CAC  relative 
to  the  ceiling  and  floor  remained  the  same,  and  if  the 
same  EAC  was  the  most  accurate  predictor  of  the  CAC,  then 
the  results  were  not  sensitive  to  Service. 

e.  System  Type  (Airframe,  Electronics,  Engine, 
Equipment,  Ground,  Missile,  Ship  or  Space) 

This  sensitivity  was  tested  by  determining  the 
minimum  and  maximum  average  percent  deviations  from  the 
CAC  for  each  System  Type.  If  the  results  agreed  with  the 
original  results,  if  the  position  of  the  CAC  relative  to 
the  ceiling  and  floor  remained  the  same,  and  if  the  same 
EAC  was  the  most  accurate  predictor  of  the  CAC,  then  the 
results  were  not  sensitive  to  System  Type. 

f.  Major  Baseline  Changes  (Not  0TB  or  0TB) 

This  sensitivity  was  tested  by  determining  the 

minimum  and  maximvim  average  percent  deviations  from  the 
CAC  for  those  contracts  which  went  Over  the  Target 
Baseline  (0TB)  and  those  that  did  not  go  Over  the  Target 
Baseline  (NOT  0TB) .  If  the  results  agreed  with  the 
original  results,  if  the  position  of  the  CAC  relative  to 
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the  ceiling  and  floor  remained  the  same,  and  if  the  same 
EAC  was  the  most  accurate  predictor  of  the  CAC,  then  the 
results  were  not  sensitive  to  Major  Baseline  Changes. 

g.  Management  Reserve 

This  sensitivity  was  tested  on  all  contracts  where 
Management  Reserve  (MR)  represented  less  than  30  percent 
of  Total  Allocated  Budget  (TAB) .  MR  should  only  be  a 
small  portion  of  TAB.  If  MR  is  greater  than  30  percent  of 
TAB,  then  the  MR  data  is  probably  incorrect  and  should  not 
be  used  in  the  analysis.  This  restriction  provided  239 
contracts  and  3508  DAES  reports  with  which  to  test  the 
sensitivity.  The  calculations  involved  subtracting  MR 
from  TAB  to  arrive  at  Budget  at  Completion  (BAC) .  BAC 
then  replaced  TAB  in  the  basic  EAC  equation.  The  minimum 
and  maximum  EAC  average  percent  deviations  from  the  CAC 
were  determined,  as  well  as  the  relative  position  of  the 
CAC  to  the  minimum  and  maximum.  Additionally,  the  EAC 
with  the  average  percent  deviation  from  the  CAC  closest  to 
zero  was  the  most  accurate  predictor  of  CAC.  If  the 
results  agreed  with  the  original  results,  if  the  position 
of  the  CAC  relative  to  the  ceiling  and  floor  remained  the 
same,  and  if  the  same  EAC  was  the  most  accurate  predictor 
of  the  CAC,  then  the  results  were  net  sensitive  to 
Management  Reserve. 
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6.  Are  the  original  results  and  the  results  in  each 
category  sensitive  to  the  Contract  Conqpletion  Stage? 

This  question  was  answered  by  considering  the 
contracts  in  each  category  during  the  three  previously 
defined  Contract  Completion  Stages.  For  example,  the 
contracts  that  were  Pre-Production  were  analyzed  at  Early, 
Middle  and  Late  Contract  Con^letion  Stages .  For  each 
conpletion  stage,  there  was  an  EAC  floor,  an  EAC  ceiling 
and  -in  EAC  which  was  the  most  accurate  predictor  of  the 
CAC.  If  the  results  based  on  Early,  Middle  and  Late 
Contract  Completion  Stages  agreed  with  the  original 
results,  then  the  Pre-Production  category  was  not 
sensitive  to  the  Contract  Cort^letion  Stage. 

The  methodologies  outlined  for  the  investigative 
questions  two  through  four  provided  the  means  to  determine 
the  original  hypothesis  test  results.  The  methodologies 
for  investigative  questions  five  and  six  provided  a  way  to 
evaluate  sensitivity  of  the  original  results  to  Index 
Type,  Program  Phase,  Contract  Type,  Service,  System  Type, 
Major  Baseline  Changes,  Management  Reserve  and  Contract 
Conpletion  Stage. 

The  results  to  these  five  remaining  investigative 
questions  are  presented  in  the  following  chapter. 
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Summary 


This  chapter  reintroduced  the  hypothesis  and 
describes  the  database  used  to  test  the  hypothesis .  The 
process  used  to  prepare  the  database  for  use  included 
reference  to  the  descriptive  fields  and  the  numeric  fields 
used  for  the  analysis.  An  overview  of  the  hypothesis  test 
provided  an  explanation  of  the  technique  used  to  evaluate 
the  hypothesis  and  sensitivity  to  the  various  conditions, 
as  described  in  the  five  remaining  investigative 
questions . 

The  equations  for  the  nine  indexes  and  EACs  were 
provided,  followed  by  a  description  of  the  methodology 
applied  to  answer  each  of  the  five  remaining  investigative 
questions . 

The  following  chapter  focuses  on  the  results  of  the 
hypothesis  test.  It  presents  the  results  to  the  five 
remaining  investigative  questions.  Graphs  of  the  overall 
results  are  provided  with  comments  and  analysis. 
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IV.  Reaulba 


This  chapter  focuses  on  the  results  of  the 
methodology.  The  hypothesis  is  restated  and  the  results 
are  presented.  The  final  five  Investigative  Questions  are 
answered,  followed  by  a  description  of  trends  noticed  in 
the  graphs  of  the  EAC  ceiling  and  floor  for  each  category 
of  contracts. 

Hypotheaia 

The  hypothesis,  as  stated  previously,  is  the 
following; 

Ho;  The  Cost  at  Completion  is  bounded  below  by  the 
CPI-based  EAC  and  above  by  the  SCl-based  EAC. 

Recall  from  the  methodology  that  several 
investigative  questions  were  posed  to  test  the  hypothesis. 
These  questions  are  answered  in  the  following  section. 

The  results  to  the  hypothesis  test  and  the 
sensitivity  analysis  are  presented  in  Table  3.  Results  to 
Contract  Completion  Stage  sensitivity  are  presented  in 
Table  4  and  Table  5.  The  graphs  of  the  EAC  ceiling  and 
floor  and  the  corresponding  index  graphs  for  each  category 
are  presented  in  Appendix  D. 
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TABLE  3 

HYPOTHESIS  TEST  RESULTS  BY  CATEGORY 


Contract  Category 
(i  of  coatraeta,  i  of  reporta)^ 

Range  of  Iadex*Baaad  EACt  ' 

Mia 

Max 

Boaada 

CAC7> 

Cloteat 
to  CAC< 

All 

Coobacta  Taken  at 

CPI 

sa 

NO 

sa 

a  Whole  f321.  3M31 

Prograa 

Pra-Produotton  (llZ  1434) 

cn 

HjS 

Pkaae 

ProductioQ  (135.  1398) 

CPI 

1^91 

Contract 

Coat  Plot  (137.  1630) 

mnm 

Type 

mfU 

HIH 

nimPUtd  (13,96) 

■SH 

Rxad  Price  (169.2033) 

Breach  of 

Army  (48,606) 

wKjm 

sa 

Seraice 

■■■ 

Navy  (144.  1424) 

l!9 

sn 

OSAP  (127. 1638) 

sa 

Syatea  Typt 

Anframa  (22,  228) 

SPI 

sa 

NO 

sa 

Eltctioaica  (43,  340) 

CPI 

sa 

NO 

sa 

Engine  (12.  117) 

CM 

sa 

NO 

CPI 

B(|ui{imaot  (19.  213) 

CPI 

sa 

NO 

sa 

Qni^  (8,86) 

cn 

sa 

NO 

CH 

Mittilet  (43.  313) 

CPI 

sa 

NO 

so 

Ships  (19,  244) 

SPI 

sa 

NO 

SPI 

SDeca(23.  339) 

cn 

sa 

NO 

sa 

Major 

Not  Otr  Target 

CP! 

sa 

NO 

sa 

Baaellae 

BateUnc  (233,3331) 

Chaagea 

Over  Ttogat  Bataliat 

SPI 

sa 

NO 

sa 

(68.  232) 

1  BACmACWP  4^  (TAB-BCWPyiotex.  wbare  isdu  it  om  of  tbMt  One  indexat: 

Cott  PMrfoimtoco  Indtx  (CPI)bBCWP/aCWP 
Sctedolt  PufocBtiiot  Indn  (SPI)-BCWP/BCWS 
Sctednli  Cott  iDdn  (Sa)i<7I*sn 

2  iofrtpotuntafloditHiiipltti»ofDAES<|iiKlHiynpo(t(latt  '*'pa(it  ■«  a  tubtet  of 
oumtiBU.  Tilt  ton  of  oooiraa  tnbeattforitf  it  hat  tbao  tiit  ntanbar  of  ooDttach  becauit 
tooit  fttUt  in  lilt  tiitabitt  wart  tapty  for  umt  cooeaeta. 

3  Do«  iha  nnga  of  iodm-batod  BACt  •vtbitiad  bonad  CAC7  CAC  (tfan  to  Coat  at  Complttioa, 
daao«lMll»ftiialACWPineaBfaa»«aGt  IfIboMINEACittowarihaaCACaadiftfaaMAX 
BAC  it  Ugbar  tban  CAC  dtao  ibt  antwor  it  YES. 

4  Tilt  BAC  clotMt  to  tba  CAC  h  dw  mott  aBCuatt  pradkaor  of  CAC 
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TABLE  4 

CONTRACT  COMPLETION  STAGE  SENSITIVITY  BY  CATEGORY 


Coatrsct 

Catacorv 

Raaga  of  ladn-Baaad  BACa* 

CoBpUtloa  SUga' 

Mia 

Mat 

BooHda 

Cloaaat  to 

(i  of  eoBtra«t8«  # 

CAC?« 

CAC» 

of  roBortsV* 

All 

malt 

Biriy  (321. 302) 

CP! 

sa 

NO 

sa 

MMdla  (321. 1073) 

CP! 

sa 

NO 

sa 

Lata  (321.  2428) 

SP! 

so 

NO 

sn 

Profran 

Phaaa 

Enly  (112. 100) 

CP! 

sa 

NO 

sa 

Mkldte  (11Z406) 

CPI 

sa 

NO 

sa 

Uia  (11Z948) 

SPI 

sa 

NO 

SPI 

2Bi^BSjjOA 

Billy  (135. 113) 

CPI 

sa 

NO 

sa 

Mkldla  (135.354) 

<3*1 

sa 

NO 

sa 

Lite  (135.931) 

sn 

sa 

NO 

sn 

Coatraet 

Ca«t  PtiM 

Typa 

Btdy  (137.90) 

CPI 

sa 

NO 

sa 

MUdla  (137.466) 

CPI 

sa 

NO 

sa 

Liii  (U7, 1074) 

SPI 

sa 

YES 

sa 

E«ly  (0.5) 

sn 

sa 

NO 

sn 

Middla  (13.25) 

cn 

sn 

NO 

cn 

Ltfi  (13.66) 

CP! 

sn 

NO 

cn 

Bariy  (169.2053) 

CP! 

sa 

NO 

sa 

MUdIa  (169. 195) 

CP! 

sa 

NO 

sa 

Lm  (169.578) 

CPI 

sa 

NO 

cn 

Braacb  of 

Sarvlca 

Ana 

Birty  (48,41) 

m 

sa 

YES 

sa 

Middla  (48.17) 

CFl 

sa 

NO 

sa 

LiB  (48. 394) 

sn 

sa 

NO 

sa 

Niw 

Bidy  (144. 112) 

sn 

sa 

NO 

sa 

Middla  (144,411) 

sn 

sa 

NO 

sa 

Lala  (144.  1001) 

sn 

sa 

NO 

SPI 

IISAP 

Barty  (127. 148) 

CPI 

sa 

NO 

sa 

Middla  (127. 471) 

cn 

sa 

NO 

sa 

Laia  (127.  1019) 

sn 

sa 

NO 

sn 

Major 

Nat  Qyg  Trot 

Baaallaa 

Bualjna 

Chao(«ia 

Bariy  (253. 272) 

cn 

sa 

NO 

sa 

Middle  (253.993) 

0*1 

sa 

NO 

sa 

Laia  (253.  2286) 

sa 

NO 

sn 

Ovar  Tiwat  BaaaHna 

Bariy  (68. 30) 

CPI 

sa 

YES 

sn 

Middla  (68,  80) 

sn 

sa 

NO 

sa 

Ute  (68. 142) 

sn 

sa 

NO 

sa 
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TABLE  4  (Continued) 


CONTRACT  COMPLETION  STAGE  SENSITIVITY  BY  CATEGORY 


1  BAC^CWP (TAB'BCWPyiDdM,  wtMi*  iadn  i«  OM  of  tbtM  ttam  indaxM: 

Coot  lodn  (CFI)bBCWP/ACWP 

SohMlol*  PMomiaim  Indn  (SFI>-BCWP/^^ 

Setednk  Cott  lodn  (SCI)«CFI*sn 

2  OoaipMoaStag«i*hnlotaeMtCaaiplM(PC)wlMf«fOBCWP/TAB. 

Birir  PC«>JS 
KOddU:  JS<PCO.70 
Uto:  FO.70 

3  fofrepcronlimlottMMi^iloiiaofDABSqiMntrty  npoftdaia.  Ropoits  «•  a  tnbtot  of 
ooomoia.  Tba  turn  of  eoninct  «ubcata|odaa  it  Im  thtt  tba  total  ranbar  of  oontracta  bacausa 
aoma  Aalda  in  tba  databata  wan  amptjr  fbr  mna  cootraett. 

4  Doat  tba  nnga  of  iodax-baatd  BACt  avaloatad  bound  CAC?  CAC  itfm  to  Coat  at  Complattoii. 
daflnad  u  tba  float  ACWP  fat  aacb  oonttacL  IftUMRlEACialoworlbaDCACaodiftbaMAX 
BAC  ia  hi(bar  than  CAC,  tbao  tba  anaarar  it  YES. 

5  Tba  BAC  chweat  to  tba  CAC  it  tba  moit  accunta  pndictor  of  CAC. 


TABLE  5 


CONTRACT  CO  .uETION  STAGE  SENSITIVITY  BY  SYSTEM  TYPE 


SyaUB  Tjp* 

Raa«e  of  ladex-BaaMl  BACe  > 

CoaplattM  SUf** 

(i  of  coatraett,  4  of 
rteorta)^ 

Mia  Mm 

Boaade  Cloeeet  to 
CAC?<  CAC* 

Early  (Zt,  18) 

MkkOe  (22.55) 

Lata  (22. 155) 

sn  sa  NO  sa 

sn  sa  YES  sa 

CPI  sa  NO  (m 

Elactronlei 

Early  (43. 43) 

Middle  (43, 160) 

Lau  (43.337) 

CPI  sn  NO  spi 

cn  sa  NO  sa 

CPI  sa  NO  CPI 

Early  (IZ  3) 

Middle  (1Z31) 

Late  (IZSn 

CPI  sa  YES  CPI 

CPI  sa  NO  sa 

SPI  sa  NO  sn 

Early  (19. 22) 

Middle  (19.  SS) 

Late  (19. 136) 

cn  sa  NO  sa 

cn  sa  NO  sa 

cn  sa  NO  cn 

Early  (8.7) 

Middle  (8.30) 

Ui*A59) 

cn  sa  NO  sa 

cn  sa  NO  sa 

sn  sa  NO  sa 

Early  (43.40) 

Middle  (43. 132) 

Lau  (43. 323) 

cn  sa  NO  sa 

cn  sa  NO  sa 

sn  sa  YES  sn 

Early  (19.24) 

Middle  (19.77) 

Late  (19. 143) 

sn  cn  NO  cn 

cn  sa  NO  sa 

sn  sa  NO  sn 

SqISI 

Early  (23.31) 

Middle  (25.91) 

Late  (23.213) 

cn  sa  YES  sa 

cn  sa  NO  sa 

sn  sa  NO  sa 
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TABLE  5  (Continued) 


CONTRACT  COMPLETION  STAGE  SENSITIVITY  BY  SYSTEM  TYPE 


1  EAC>iACWP4-(TABBCWPyiiiikx,«taaniitdnitOfMoftlMMttmiadcw«: 

Cott  PwfbmaDn  Ittdtt  (CFI>i9CWp/ACWP 
Soteauto  Itaftaounn  iBte  (SPIHB^ 

SctMdnto  p»t  Indn  (SCIHCn*sn 

2  Coeipiiiain  Sm»  whw  to  PwotatCompktt  (K)  wlww  W:»BCWP/TAB. 

Barir.  FC<(^ 

MkhllK  35<PC«i.70  j 

tm:  VO.IO 

3  •ofnpQrtsrafimtDthaMnpteaiaiotl'AE.^  r.<«ttriyi«pQitdaiL  iUpoitt  «  •  nbiM  of 
ooiunee.  Tho  aomofconoocfbootogoriw  i»  lamb—  dio  tool  oombw  of  oomroctt  boonao 
aoiM  fiolda  in  iho  datibiaa  won  mptf  fat  aone  oaotracta. 

4  DoMdMrangoofiadtx-bMtdEACaawifauMdboaadCAC?  CAC  mfan  to  Coat  at  Cooiplotioo. 
daflnad  ••  tbs  final  ACWP  in  aachfloiniDt  If  tbs  MIN  EAC  ia  knaar  toan  CAC  and  if  lbs  MAX 
EAC  iabifibsr  than  CACtbaatfafaiasria  YES. 

5  ‘IbsEACeiosasttoibbCACia  tosmoataecanispndiGiorofCAC. 
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Inveatlaative  Queationa  and  Hypothesis  Teat  Results 

The  first  investigative  question  was  answered  in 
Chapter  Two  with  the  review  of  £AC  research  to  date.  The 
remaining  questions  are  answered  here.  The  questions  are 
restated  as  they  appear  in  Chapter  One  and  Chapter  Three, 
followed  by  a  discussion  of  the  results  and  an  answer  to 
each  question. 

2.  Of  the  index-based  EACs  compared,  is  the  CPI -based  EAC 
the  floor  and  is  the  SCI-based  EAC  the  ceiling? 

Of  the  EACs  compared,  the  CPI-based  EAC  was  the 
minimum  and  the  SCI-based  EAC  was  the  maximum.  This 
confirmed  the  CPI-based  EAC  as  the  floor  and  the  SCI-based 
EAC  as  the  ceiling  for  the  entire  set  of  contracts. 

OVIll-ALL  FLOORS  CFZ-BA8BD  BAC 
OVBR-ALL  CBZLIMOx  8CZ-BA8BD  BAC 

3.  Does  the  final  Cost  at  Completion  lie  within  the 
considered  reuige  of  EACs? 

The  final  CAC  did  not  lie  within  the 
considered  range  of  BACs.  On  average,  the  CAC  was 
slightly  higher  than  the  range  of  EACs. 

4.  Which  EAC  is  the  most  accurate  predictor  of  final  cost 
at  Completion? 
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Th«  8CI>b«««d  SAC  w«s  tb«  most  aocurato 
prmdlotor  of  tbo  final  CAC.  On  average,  the  SCI -based 
EAC  underestimated  the  CAC  with  an  average  percent 
deviation  from  CAC  of  -1.01  percent. 

5.  Are  the  original  hypothesis  test  results  (cmrwers  to 
questions  two  through  four)  seiiisitive  to: 

a.  Index  Type  (Cumulative,  Six-Month  or  Three-Month) 
The  cumulative  CPI -based  EAC  and  the  cumulative  SCl-based 
EAC  bounded  the  cumulative  SPl-based  EAC.  The  six-month 
and  three-month  results  had  the  same  EAC  rank  order  as  the 
cumulative  results.  Since  these  results  for  each  Index 
type  agreed  with  the  original  results,  the  original 
results  were  not  sensitive  to  the  index  Type. 

Another  inportant  observation  resulted  from  analysis 
of  the  EAC  average  percent  deviations  from  the  CAC 
according  to  Index  Types.  The  cumulative  CPI,  six-month 
CPI  and  the  three-month  CPI  all  tended  to  be  very  close 
together.  Likewise,  the  cumulative,  six-month  and  three- 
month  values  for  SPI  were  close  together  and  the 
ounulative,  six-month  and  three  month  SCI  values  were 
close  together. 

As  explained  in  the  justification  for  the  methodology 
in  Chapter  Three,  it  was  not  appropriate  to  apply  a  t-test 
on  the  signific2mce  of  the  difference  between  means  across 
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the  EACs.  However,  it  did  seem  apparent  by  visual 
inspection  that  the  CPIs  are  close  to  each  other,  SPIs  are 
close  to  each  other  and  SCIs  are  close  to  each  other, 
while  the  groups  of  CPIs,  SPIs  and  SCIs  all  seem  to 
estimate  in  three  distinct  groups. 

t 

b.  Program  Phase  (Pre-Production  or  Production) 

Both  Pre-Production  and  Production  contracts  had  the  same 
ceiling  and  floor  as  the  original  results.  The  CAC  was 
bounded  by  the  floor  and  ceiling  for  Production  contracts 
only.  The  most  accurate  predictor  of  the  CAC  for  Pre- 
Production  contracts  agreed  with  the  original  results 

(SC I -based  EAC) .  The  most  accurate  predictor  of  the  CAC 
for  Production  contracts  was  the  SPl-based  EAC.  Thesa 
findings  indicated  that  the  original  results  were 
sensitive  to  the  Program  Phase. 

c.  Contract  Type  (Cost  Plus,  Firm  Fixed  or  Fixed 
Price) 

The  floor  and  ceiling  for  all  three  Contract  Types  agreed 
with  the  original  results.  The  position  of  the  CAC 
relative  to  the  ceiling  and  floor  differed  among  the  three 
contract  types. 

The  ceiling  and  floor  for  Cost  Plus  contracts 
underestimated  the  CAC  and  had  the  SCI-based  EAC  for  the 
most  accurate  predictor  of  the  CAC.  The  ceiling  and  floor 
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for  Firm  Fixed  contracts  overestimated  the  CAC  and  had  the 
CPI*based  EAC  for  the  most  accurate  predictor  of  the  CAC. 
The  ceiling  and  floor  for  Fixed  Price  contracts  bounded 
the  CAC  and  had  the  SCI*based  EAC  for  the  most  accurate 
predictor  of  CAC.  These  findings  indicated  that  the 
original  results  were  sensitive  to  the  Contract 
Type . 


d.  Service  (Anny,  Navy  or  USAF) 

The  results  for  the  Army  and  USAF  contracts  agreed  with 
the  original  results.  The  Navy  results  indicated  the 
floor  was  the  SPI -based  EAC  and  the  ceiling  agreed  with 
the  original  results.  The  Navy  ceiling  and  floor  both 
slightly  overestimated  the  CAC  and  the  SPl-based  EAC  was 
closest  the  CAC.  These  findings  indicated  that  the 
original  results  were  sensitive  to  the  Branch  of 
Service. 


e.  System  Type  (Airframe,  Electronics,  Engine, 
Equipment,  Ground,  Missile,  Ship  or  Space) 

Results  of  this  sensitivity  agreed  with  the  original 
results,  with  the  following  exceptions.  The  floor  for 
Airframe  contracts  was  the  SPI-based  EAC.  The  ceiling  and 
floor  bounded  the  CAC. 

The  ceiling  and  tloor  for  both  Engines  and  Equipment 
agreed  with  the  original  findings  but  they  overestimated 
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the  CAC,  making  the  CPI-based  EAC  the  most  accurate 
predictor  of  CAC. 

The  floor  for  Ships  was  the  SPI-based  EAC  and  the 
ceiling  and  floor  overestimated  the  CAC,  making  the  SPI- 
based  EAC  the  most  accurate  predictor  of  the  CAC.  These 
findings  indicated  that  the  original  results  were 
sensitive  to  the  System  Type. 

f.  Major  Baseline  Changes  (Not  OTB  or  0TB) 

The  Over  Target  Baseline  contracts  had  the  SPI-based  EAC 
floor.  This  indicated  that  results  were  sensitive  to 

Major  Baseline  Changes. 

g.  Management  Reserve 

The  results  with  MR  removed  from  the  calculations  agreed 
with  the  original  results,  suggesting  that  the  original 
results  were  not  sensitive  to  Management  Reserve. 

6.  Are  the  original  results  and  the  results  in  each 
category  sensitive  to  the  Contract  Con^letion  Stage? 

The  emswers  to  this  question  are  briefly  sximmarized  below 
and  are  presented  in  Table  4  and  Table  5. 

Original  Results:  The  original  results  remained  the 
same  in  the  Early  and  Middle  Contract  Completion  Stages. 
The  SPI-based  EAC  became  the  floor  in  the  Late  Stage,  with 
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the  SPI-based  EAC  as  the  most  accurate  predictor  of  the 
CAC.  These  findings  indicate  that  the  original  reeulte 
were  sensitive  to  the  Contract  Completion  stage. 

Program  Phase:  The  Pre-Production  and  Production 

results  differed  from  the  original  results  in  Che  Late 
Conpletion  Stage.  Th^  had  the  SPI-based  EAC  floor  and 
the  SPI-based  EAC  was  the  most  accurate  predictor  of  the 
CAC  in  both  cases.  These  results  suggest  that  Program 
Phase  results  were  sensitive  to  the  Contract 
Completion  Stage. 

Contract  Type:  Cost  Plus  and  Firm  Fixed  Contracts  all 

showed  slightly  different  results  across  the  Early,  Middle 
and  Late  Completion  Stages.  Cost  Plus  had  the  SPI -based 
EAC  as  a  floor  in  the  Late  Completion  Stage  while  Firm 
Fixed  Contracts  had  the  SPI-based  EAC  as  a  floor  in  the 
Early  Con^letion  Stage.  These  findings  suggest  that 
Contract  Type  results  were  sensitive  to  the 
Contract  Completion  Stage. 

Service:  The  Army  Contracts  started  with  the  SPI-based 
EAC  floor  and  switched  to  the  CPi-based  floor  in  the 
Middle  Completion  Stage.  Air  Force  Contracts  started  with 
the  CPI-based  EAC  floor  then  switched  to  the  SPI-based 
floor  in  the  Late  Completion  Stage.  These  findings 
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suggest  that  the  Branch  of  Sexvioe  results  were 
sensitive  to  the  Contract  Cospletlon  Stage. 

Systen  Type:  The  SPI-based  £AC  was  the  floor  in  the 
Early  Conqpletion  Stage  for  Ships  and  in  the  Early  and 
Middle  Stages  for  Airframes.  The  SPI*hased  EAC  was  the 
ceiling  for  Electronics  Contracts  in  the  Early  stage.  The 
CPI-based  EAC  was  the  ceiling  for  Ships  in  the  Early 
stage.  These  results  suggest  that  System  Type  results 
were  sensitive  to  the  Contract  Completion  Stage. 

Major  Baseline  Changes:  The  floor  for  contracts 

with  no  major  baseline  changes  (not  0TB)  was  the  SPI-based 
EAC  in  the  Late  Cospletlon  Stage.  The  SPI-based  EAC  was 
also  the  floor  for  0TB  contracts  in  the  Middle  and  Late 
Conpletion  Stages.  These  results  suggest  that  the 
results  based  on  Major  Baseline  Changes  were 
sensitive  to  the  Contract  Coapletion  Stage. 

In  addition  to  anwering  the  investigative  questions 
by  euialyzing  the  EAC  average  percent  deviations  from  the 
Cost  at  Completion,  there  were  several  noteworthy  trends 
appeirent  in  the  graphs  of  the  indexes  and  the  EAC  average 
percent  deviations  from  the  CAC. 
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Graphical  Analvala  of  Indexes 


The  indexes  followed  a  predictable  pattern,  with  the 
exception  of  the  50  to  60  percent  completion  point.  There 
was  a  pronounced  upward  spike  in  SPI  and  SCI  for  All 
Contracts,  Production,  Fixed  Price,  Navy  and  0TB 
Contracts . 

Further  analysis  revealed  one  Navy  contract  which 
had  extremely  high  SPI  values  between  50  and  60  percent 
complete.  This  one  contract  caused  the  upward  spike  in 
the  SPI  and  the  SCI.  This  increase  in  the  SPI  and  the  SCI 
was  not  large  enough  to  noticeably  affect  the  EAC  average 
percent  deviations  from  the  CAC. 

Figxires  1,  2,  3  and  4  illustrate  the  overall  results. 
Figures  1  and  2  represent  the  overall  EAC  ceiling  and 
floor  and  the  corresponding  indexes  with  this  Navy 
contract  included.  Figures  3  and  4  represent  the  same 
overall  results,  with  this  Navy  contract  removed. 
Coii¥>arlson  of  Figures  1  and  2  to  Figures  3  emd  4  revealed 
that  this  one  Navy  contract  had  a  significemt  intact  on 
the  graph  of  the  indexes  but  it  did  not  significantly 
affect  the  graph  of  the  EAC  ceiling  and  floor. 
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Figure  1.  Overall  EAC  Ceiling  amd  Floor 


Figure  2.  Overall  Index  Ceiling  and  Floor 


Figure  3 .  Overall  (Navy  Contract  Removed)  EAC  Ceiling  and 

Floor 


Figure  4.  Overall  (Navy  Contract  Removed)  Index  Ceiling 

and  Floor 
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Graphical  Analysis  of  EACa 


Analysis  of  the  EAC  graphs  revealed  several 
noteworthy  trends  among  various  contract  categories,  as 
described  below. 

1.  EACs  tended  to  estimate  the  CAC  quite,  accurately  prior 
to  the  25  percent  completion  point.  Between  25  and  65 
percent,  the  EACs  took  a  fairly  pronounced  downward  dip, 
then  approached  the  CAC  at  the  75  percent  completion 
point.  The  EACs  slightly  overestimated  the  CAC  between 
the  75  and  95  percent  completion  points.  This  "dipper" 
effect  ocaired  among  all  the  contracts  taken  as  a  whole, 
Pre-Production,  Cost  Plus,  Army,  Over  Target  Baseline  and 
Not  Over  Target  Baseline  contracts. 

2.  Several  contract  categories  estimated  fairly  close  to 
the  CAC  throughout  the  entire  contract  life.  These 
included  Production,  Navy,  Air  Force,  and  Missile 
contracts. 

3.  A  few  categories  overestimated  the  CAC  in  the  Early 
Completion  Stage,  then  underestimated  the  CAC  between  35 
and  75  percent  completion  and  finished  in  the  Late  Stage 
close  to  CAC.  These  included  Engines,  Equipment  and  Space 
contracts. 
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4.  There  were  also  some  categories  which  estimated  low  in 
the  Early  Stage  then  recovered  to  estimate  close  to  the 
CAC  in  the  Middle  and  Late  Stages.  This  trend  occured  in 
Airframe,  Electronics  and  Ship  contracts. 

5.  Firm  Fixed  contracts  tended  to  overestimate  the  CAC 
throughout  the  entire  contract,  while  Fixed  Price 
contracts  tended  to  underestimate  the  CAC. 

Summary 

This  chapter  answered  the  remaining  investigative 
questions  and  presented  results  of  the  hypothesis  test. 
Overall,  the  CPI-based  EAC  and  the  SCI-based  EAC  were 
confirmed  as  the  boundaries  to  the  range  of  EACs.  The 
SCI-based  EAC  was  confirmed  as  the  most  accurate  predictor 
of  the  Cost  at  Completion.  These  overall  results  were  not 
sensitive  to  Index  Type  (cumulative,  six-month  or  three- 
month)  or  Management  Reserve.  The  results  were  sensitive 
to  Program  Phase,  Contract  Type,  Breunch  of  Service,  System 
Type  and  Major  Baseline  Changes. 

Having  provided  the  results  of  the  hypothesis,  the 
final  chapter  discusses  these  results  and  their 
significance  to  program  managers.  Areas  for  future 
related  research  follow  this  discussion. 
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IL _ Diacussion 


This  chapter  summarizes  the  research  by  reviewing  the 
hypothesis  and  restating  the  conclusions  and  implications 
for  program  mcuiagers .  The  discussxon  recaps  the  data 
analysis  followed  by  recommendatic'ns  for  further  research. 

Review  of  the  Hypothesis 

This  thesis  explored  the  hypothesis  that  the  Cost  at 
Con^letion  (CAC)  is  b  unded  below  by  the  CPI-based 
Estimate  at  Completion  (EAC)  ai  above  by  the  SCI-based 
EAC.  The  hypothesis  was  tested  on  321  contracts. 
Descriptive  statistics  provided  a  method  for  testing  the 
hypothesis.  Aaditional  information  resulted  from 
analyzing  trends  in  graphs  of  the  range  of  EACs  by 
category . 

Cancluaiflp 

Of  the  EACs  evaluated,  the  CPI-based  EAC  was 
confirmed  as  the  floor  and  the  SCI-based  EAC  was  confirmed 
as  the  ceiling.  On  average,  the  range  of  EACs  tended  to 
slightly  underestimate  the  CAC,  thus  the  CAC  was  not 
bounded  by  the  range  of  EACs  as  hypothesized.  The  SCI,  on 
average,  was  the  most  accurate  predictor  of  the  CAC. 

The  results  were  tested  for  sensitivity  to  Index  Type 
(cumulative,  six-month  and  three-month) ,  Contract 
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Conpletion  Stage,  Program  Phase,  Contract  Type,  Branch  of 
Service,  System  Type,  Major  Contract  Baseline  Changes  and 
Management  Reserve.  The  results  were  sensitive  to  all  of 
these  conditions  except  Management  Reserve.  With 
Management  Reserve  removed  from  the  calculations,  the 
results  agreed  with  the  original  results. 

Analysis  of  Results 

The  hypothesis  test  methodology  consisted  of 
analyzing  descriptive  statistics  cf  the  DAES  database. 

The  mean  was  the  key  statistic  used  and  the  standard 
deviation  and  coefficient  o  ‘-iation  were  also 
calculated  to  provide  information  on  data  dispersion  for 
each  of  nine  EACs  over  3803  DAES  reports  of  information. 

The  values  for  three  and  six  month  indexes  were 
interpolated  from  existing  data  for  each  of  the  DAES 
reports.  This  was  accomplished  with  a  script  written  in 
Paradox^  Application  Language  (PAL),  presented  in  Appendix 
B. 

Upon  completion  of  interpolating  needed  values  for 
three  and  six  month  indexes,  the  EACs  were  calculated  for 
each  of  the  3803  DAES  reports. 

The  resulting  EACs  were  normalized  into  percent 
deviations  from  the  CAC.  These  percentages  were  then 
averaged  over  early,  middle,  late  and  overall  stages  of 
completion.  Additionally,  the  standard  deviations  and 
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coefficients  of  variation  were  calculated  over  these 
stages . 

The  EAC  with  the  lowest  average  percent  deviation 
from  the  CAC  was  defined  as  the  floor  and  the  EAC  with  the 
highest  average  percent  deviation  from  the  CAC  was  defined 
as  the  ceiling.  The  CAC  was  defined  to  be  bounded  by  the 
floor  and  ceiling  if  the  floor  was  less  than  the  CAC  and 
the  ceiling  was  greater  than  the  CAC. 

The  results  show  that  on  average,  the  EACs  are 
bounded  below  by  the  CPI-based  EAC  and  above  by  the  SCI- 
based  EAC.  Because  the  range  of  EACs  tended  to 
underestimate  the  CAC,  the  CAC  was  not  bounded  by  the 
floor  and  ceiling.  However,  it  is  important  to  note  that 
for  the  contracts  overall,  the  range  of  EACs  had  an 
average  percent  deviation  from  the  CAC  of  less  than  five 
percent.  This  made  the  CPI,  SPI  and  SCI -based  EACs  very 
close  predictors  of  the  CAC,  despite  the  fact  that  the  CAC 
was  not  actually  bounded  by  them. 

Analysis  of  the  EACs  over  different  contract 
completion  stages  by  category  resulted  in  EAC  average 
percent  deviations  from  the  CAC  which  ranged  from  -17 
percent  to  -t-30  percent.  In  addition  to  the  range,  an 
important  observation  involved  the  groupings  of  each 
index . 

The  CPIs  were  fairly  close  to  each  other,  the  SPIs 
were  fairly  close  to  each  other  and  the  SCIs  were  fairly 
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close  to  each  other.  This  had  important  in^lications . 
Although  the  six-month  CPI -based  EAC  was  the  actual  EAC 
floor,  it  differed  from  the  cumulative  CPI -based  EAC  by 
less  than  one  percent  average  deviation  from  the  CAC. 

This  made  the  cuxnulative  CPI-based  EAC  a  very  close 
approximation  of  the  floor  for  EACs.  The  cumulative  CPI 
is  much  easier  to  ascertain  from  report  data  because  it 
requires  only  current  information.  The  three  and  six- 
month  EACs  require  several  consecutive  months  of  previous 
report  data,  which  may  not  always  be  available. 

The  bottom  line  for  program  managers  is  that  the 
cumulative  CPI-based  EAC  and  the  cumulative  SCI-based  EACs 
are  very  close  approximations  of  the  floor  and  ceiling  for 
EAC  formulas.  Overall,  the  actual  CAC,  on  average,  is  not 
bounded  by  the  range  of  EACs.  The  SCI-based  EAC  is  the 
most  accurate  predictor  of  the  CAC. 

The  graphs  of  the  EAC  ceiling  and  floor  for  each 
category  show  the  program  metnagers  how  the  remge  of  EACs 
relates  to  the  actual  Cost  at  Conqpletion.  These  graphs 
should  help  program  analysts  predict  reasoned^le  conviction 
costs  for  different  categories  of  contracts. 

For  example,  an  analyst  working  on  a  missile  contract 
estimate  could  refer  to  Figure  5  the  graph  for  Missile 
Contracts.  If  the  contract  was  in  the  Early  Completion 
Stage,  the  analyst  would  know  that  the  SCI-based  EAC  would 
provide  the  most  accurate  predictor  of  the  final  Cost  at 
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Con^letion.  The  analyst  would  also  know  that  an  estimate 
lower  than  the  CPI -based  EAC  would  be  overly  optimistic. 


Figure  5.  Missile  EAC  Ceiling  and  Floor 


The. arrows  on  tba  taorixontal  axis  (X  axis-Percent 
Coaqplete)  represent  transition  points  between  Early, 
Middle  ^lnd  Late  Contract  Completion  Stages.  Specifically 


Early;  Contract  Percent  Coin)lete  <  .35 
Middle;  .35  <  Contract  Percent  Complete  <=  .70 
Late:  Contract  Percent  Complete  >  .70 


Areas  for  Future  Reagargh 


One  limitation  of  this  research  effort  is  that  the 
Management  Reserve  sensitivity  analysis  was  not  extended 
to  the  individual  categories  at  early,  middle  and  late 
stages  of  conpletion  to  see  if  eliminating  MR  might  have 
made  a  difference  in  one  or  more  of  tnese  isolated 
circumstances . 

Additional  sensitivity  analysis  might  also  prove 
useful  by  further  stratifying  the  categories.  For 
exeui^le,  a  program  manager  might  wish  to  make  inferences 
about  a  Missile  program  which  is  Pre-Production  and  Cost 
Plus.  Further  sensitivity  emalysis  on  putting  the  data 
into  constant  dollars  by  eliminating  inflation  from  the 
data  would  strengthen  the  final  results. 

Finally,  a  more  detailed  statistical  analysis  of  the 
DAES  database  would  be  very  useful  for  future  research. 
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Pefini&iQnfl 


This  Appendix  presenea  photocopied  pages  from  DOD 
5000. 2-H  displaying  official  C/SCSC  definitions  of  terns 
\ised  in  this  thesis.  Also  presented  are  definitions  of 
terns  not  included  in  DOD  5000. 2-M  but  relevant  to  this 
research  effort. 
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Feb  23,  91 
5000.2,  PARni 
SECTION  8 
AnACHMENT2 


COST/SCHEDULE  CONTROL  SYSTEMS  OEFINITIONS 


1.  Actual  Coat  of  Work  Perforwd  (ACWP).  Th«  coat  incurred  and  r«eord«d  In 
aeeonpilshing  the  worte  parforaad  within  a  given  tiaa  period. 

2.  Actual  Dtrret  C«i3ta.  Those  costa  identified  specifically  with  a 
contract,  based  upon  the  contractor's  cost  identification  and 
accuanilation  syutem  as  accepted  by  the  cognisant  Defense  Contract  Audit 
Agency  representatives.  (See  definition  14,  below.} 

3.  Allocated  Budget.  (See  definition  32,  below.) 

4.  Annlled  Direct  Cost.  The  anoust  recognised  in  the  tisM  period 
associated  with  the  consuoption  of  labor,  oaterial,  and  other  direct 
resources,  without  regard  to  the  dace  of  eoar  taent  or  the  date  of 
payment .  These  aoounts  are  to  be  charged  to  worlc-in>progress  in  the 
tine  period  chat  any  one  of  the  following  occurs: 

a.  When  labor,  eater ial,  and  ocher  direct  resources  are  actually 
consuoed. 

b.  When  eater U  1  resources  are  withdrawn  froe  inventory  for  use. 

c.  When  eater  if  1  resources  are  received  Chat  are  Identified  uniquely  to 
Che  contract  and  scheduled  for  use  within  60  days. 

d.  When  major  components  or  assemblies  are  received  on  a  line  flow 
besis  that  are  identified  specifically  and  uniquely  to  a  single 
serially  numbered  end  item. 

5.  Apportioned  gff<»rt.  Effort  that  is  not  readily  divisible  into  work 
paokages,  but  ii>  related  proportionately  to  esasured  effort. 

6.  Authorised  Work.  Effort  that  has  been  definltized  and  is  on  contraet, 
plus  that  for  which  definltized  oontraot  costs  have  net  been  agreed  bo, 
but  for  which  written  authorization  has  been  reeelved. 

7.  Baseline.  (See  definition  24,  below.) 

8.  Budgeted  Cost  for  Work  Performed  (BCWF).  The  sum  of  the  budgets  for 
completed  work  packages  and  coavleted  portions  of  open  work  paokages, 
plus  the  appllerble  portion  of  the  budgets  for  level  of  effort  and 
apportioned  effc-rt. 

9.  Budgeted  Cost  fer  Work  Scheduled  (BCWS) .  The  sum  of  budgets  for  all 
work  packages,  (lanning  packages,  etc.,  scneduled  to  be  accomplished 
(including  in>process  work  packages),  plus  Che  asMunc  of  level*of-effere 


66 


and  apport ioni-rt  effort  i‘chetJiil**d  to  be  arcom|)l  ish*r«J  wthtn  .i  Riven  time 
period . 

10.  Budgets  Tor  Work  PaokaRev-  (.Src  (It^finition  16,  Inflow  ) 

11.  Contract  Budget  Base.  The  negotiated  contract  coat  plus  the  eatlmated 
cost  of  authorized  unpriced  work. 

12.  Contractor.  An  entity  in  private  Industry  which  enters  Into  contracts 
with  the  Government.  In  this  Instruction,  the  word  also  eay  apply  to 
Government-owned,  Government -opera  ted  activities  that  perform  work  on 
defense  programs. 

13.  Coat  Account.  A  management  control  point  at  which  actual  costs  may  be 
accumulated  and  compared  to  the  budgeted  cost  of  the  work  performed.  A 
cost  account  is  a  natural  control  point  for  cost/schedule  planning  and 
control,  since  it  represents  the  work  assigned  to  one  responsible 
organizational  element  on  one  contract  work  breakdown  structure  element. 

14.  Direct  Costs.  Any  costs  that  may  be  identified  specifically  with  a 
particular  final  cost  objective.  This  term  is  explained  in  the  Federal 
Acquisition  Aegulatlon  (reference  (D). 

15.  Estimate  at  Ccmoletion  (EAC).  Actual  direct  costs,  plus  inspect  costs 
allocable  to  the  contract,  plus  estimate  of  costs  (direct  and  indirect) 
for  authorized  work  remaining. 

1$,  Indirece  coats.  Costs,  which  because  of  their  incurrence  for-  coomon  or 
Joint  Objectives,  are  not  subject  readily  to  treatment  as  direct  iiosts. 
This  term  Is  further  defined  In  the  Federal  Acquisition  Regulation 
(reference  (f)). 

17.  Initial  Budget.  (See  definition  22,  below.) 

18.  Internal  Replanning.  Replanning  actions  performed  by  the.  contractor  for 
remaining  effort  within  the  recognized  total  allocated  budget. 

19.  Level-of-Effort  (LOE).  Effort  of  a  general  or  supportive  nature  that 
does  net  produce  definite  end  products. 

20.  Management  Reserve  or  Management  Reserve  Budget.  An  amount  of  the  total 
allooated  budget  withheld  for  oanageaent  eontrol  purposes,  rather-  then 
designated  for  the  aceomplishment  of  a  specific  task  or  set  of  tasks. 

It  Is  not  a  part  of  the  performance  measurement  baseline. 

21.  negotiated  Contract  Cost.  The  estimated  cost  negotiated  in  a  cost  plus 
fixed  fee  contract,  or  the  negotiated  contract  target  cost  in  either  a 
fixed  price  incentive  ijontract  or  a  cost  plus  Incentive  fee  contract. 

22.  Original  Budret.  The  budget  t'vtabl  Ished  at,  or  near,  the  time  that  the 
contract  was  signed  and  based  on  the  negotiated  contract  cost. 

23.  Overhead .  (See  definition  i6,  .nbove.) 
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r«b  21,  91 
S000.2.  PABt  11 
SECTION  B 
ATTACNHENT  2 


24.  Kerromanea  M«»a.nurffmi‘nL  IM.^eiine.  The  time  phased  budget  plan  againet 
which  contract  perruroKince  Is  measured.  It  is  foraed  by  the  budgets 
assigned  to  scheduled  cost  accounts  and  the  applieabie  indirect  budgets. 
For  future  effort,  not  planned  to  the  cost  account  level,  the 
perforaanee  ■easureeent  baseline  also  Includes  budgets  assigned  to 
higher  level  contract  woric  breakdown  structure  elements  and 
undistributed  budgets.  It  equals  the  total  allocated  budget  less 
■anageaent  reserve. 

25.  Performlne  Qrffanization.  A  defined  unit  within  the  contractor's 
organizational  structure,  which  applies  the  resources  to  perform  Che 
work. 

26.  Planning  Package.  A  logical  aggregation  of  far  term  work  within  a  cost 
account  which  may  be  identified  and  budgeted  in  early  baseline  planning, 
but  is  not  yet  defined  into  work  packages. 

27.  Procuring  Activity.  The  subordinate  eosnamd  in  which  the  Procurement 
Contracting  Officer  Is  located.  It  may  include  the  program  offloe, 
related  functional  support  offices,  and  procurement  offices.  Examples 
of  procuring  aotlvicies  are  the  Army  Missile  Command,  Maval  Sea  Systems 
Connand,  and  Air  Force  Electronic  Systems  Division. 

28.  Replanning.  (See  definition  18,  above.) 

29.  BtnrffTrMnlPE  Replanning  of  the  effort  remaining  in  the  contract, 
resulting  in  a  new  budget  allocation  that  exceeds  the  contract  budget 
base. 

30.  ffesponsible  Oraanizsclon.  A  defined  unit  within  the  contractor's 
organizations!  structure  that  is  assigned  responsibility  for 
accomplishing  specific  tasks. 

31.  Significant  Vsrtanoes.  Those  differences  between  planned  and  actual 
performnce  requiring  further  review,  analysis,  or  action.  Thresholds 
should  be  established  as  to  the  magnitude  of  variances  that  will  require 
vmrianoe  analysis,  and  the  thresholds  should  be  revised  as  needed  to 
provide  ananingful  analysis  during  execution  of  the  contract. 

32.  Total  Alloeated  Budget.  The  sum  of  all  budgets  allocated  to  the 
contraot.  Total  alloeated  budget  eensists  of  the  perforaanee 
amasureoMnt  baseline  and  all  managemenc  reserve.  The  total  alloeated 
budget  will  reeonoile  directly  to  the  contract  budget  base.  Any 
differenocs  will  be  documented  as  to  quantity  and  cause. 

33.  Undistributed  Budget.  Budget  sppliosble  to  eentraet  effort  that  has  not 
yet  been  identified  to  contract  work  breakdown  structure  elements  at,  or 
below,  the  lowest  level  of  reporting  to  the  Government. 

3®.  Var lances ■  (See  definition  31,  above.) 

35.  Work  Breakdown  Structure  (WSS).  (See  Section  6-8.) 
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36.  Uorie  Paeteaae  Biidaets.  RMOurcos  that  are  aasignod  romvUly  *>/  the 

contractor  to  accoopliah  a  work  package,  expressed  in  dollars .  hours, 

.  standards,  or  other  definitive  units. 

37.  liarit  Packages .  DtUlled  tasks  or  material  Items  Identified  by  the 

contractor  for  accomplishing  work  reduired  to  complete  the  centract.  A 

work  package  has  the  following  characteristics: 

a.  It  represents  units  of  work  at  levels  where  work  Is  perftrmed., 

b.  It  is  cle-arly  distinguishable  from  all  other  work  jmekagts. 

0.  It  Is  assignable  to  a  single  crg-nnlzatlonal  element. 

d.  It  has  scheduled  start  and  completion  dates  as  applicable. 
Interim  milestones;  all  of  which  are  representative  of  plyalcal 
aoeomplishaent. 

e.  It  has  a  budget  or  assigned  value  expressed  in  term;  of  dollars, 
manhours,  or  other  measurable  units. 

f.  Its  duration  la  limited  to  a  relatively  short  time  jpan  tr  It  Is 
subdivided  by  discrete  value  milestones  to  case  the  objective 
measurement  of  work  performed. 

f.  It  Is  integrated  with  detailed  engineering,  manufacioirlnj;,  or  other 
schedules. 
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Budget  afe  gompletien  fBACl :  BXC  is  the  amount  planned  to 
be  spent  in  coiqpleting  the  entire  contract. 

Coat  at  gempletion  (C\r  \  :  CAC  is  the  final  cvomilative 
dollar  figure  ^ent  in  completing  the  contract.  This 
figure  is  obtained  by  taking  the  maximum  (final)  ACWP 
entry  in  each  contract. 

Pageant  goBiPlete  (PC)  :  PC  represents  the  portion  of  the 
entire  contract  that  is  con^lete  to  date.  There  are 
several  ways  to  calculate  PC.  'Hiis  thesis  defines  PC  as 
the  ratio  of  BCWP  to  TAB. 
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This  Appendix  explains  the  procedure  used  to 
interpolate  values  for  the  six^month  and  three-month 
indexes.  The  overall  process  explanation  is  followed  by 
a  copy  of  the  actual  script  used  to  interpolate  the 
values, written  in  Paradox  Application  Leuiguage  (PAL®)  . 


71 


OvcralX  Procflas 


Data  definition  and  calculations  were  accomplished  with 
personal  computer  database  and  spreadsheet  software.  The 
DAES  database  is  a  relational  database  which  lends  itself 
well  to  calculating  and  comparing  subgroups  of  data  to  test 
the  hypothesis.  While  it  is  not  necessary  to  cover  in  great 
detail  how  the  software  works,  it  does  help  to  have  a  basic 
xmdersteuiding  of  database  structure  amd  functions  in  refining 
and  calculating  data. 

A  database  is  organized  into  rows  or  records  (report 
dates,  in  this  case)  and  columns  or  fields  (the  indexes,  EACs 
and  data  elements  defined  in  Appendix  A)  .  The  basic  data  set 
never  changes.  There  are  numerous  ways  to  manipulate  the 
data,  obtain  subsets  of  the  data,  and  categorize  the  data 
through  use  of  database  tools. 

A  query  is  perhaps  the  most  useful  anc  most  powerful  of 
the  database  tools.  A  query  is  a  request  for  a  subset  of  the 
data,  based  on  criteria  established  by  the  user.  The  result 
of  a  query  is  called  the  answer  table.  Some  queries  are 
calculations  performed  on  existing  fields  while  other  queries 
are  comparison  operators,  such  as  'less  than,'  'greater  than' 
or  'equal  to.*  The  notable  attribute  of  queries  and  answer 
tables  is  that  the  original  data  remains  unaltered.  Further 
queries  can  be  performed  on  the  new  answer  tables  until  the 
user  gets  the  needed  information.  The  spreadsheet  software 
comes  is  most  appropriate  for  statistical  calculations .  When 
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used  in  conjunction  with  the  database  software,  it  is  a 
smooth  process  to  export  queried  data  between  the  database 
and  the  spreadsheet  software  for  ease  of  calculation  and 
graphing . 

IntflgpQlatiQp 

As  mentioned  in  Chapter  One,  a  shortcoming  of  the 
database  was  the  lack  of  consistent  reporting  periods  in  the 
data.  For  eight  of  the  twelve  indexes  selectee  .or 
evaluation,  monthly  data  was  necessary  for  calculating  the 
indexes  and  their  corresponding  EACs.  This  monthly  data  was 

4 

obtained  by  linear  interpolation.  While  it  is  not  known  with 
certainty  that  the  data  increased  linearly  between  submit 
dates,  this  assua^tion  is  necessary  to  obtain  the  most  recent 
three  months  of  data  and  the  most  recent  six  months  of  data. 
The  interpolation  process  is  based  on  the  notion  that  the 
ratio  of  successive  data  points  'j  the  same  as  the  ratio  of 
their  corresponding  submit  dates. 


ExMPle; 

Given  the  following  contract,  calculate  BCWPl  for 

Q2/25nr  , 

vdiere  BCWPl  « 

the  value  for  BCWP  one  month  prior  to 

02/25/78. 

80acXT  QATI 

BCNF 

07/25/77 

71 

10/25/77 

108 

02/25/78 

205 

04/25/78 

205 

08/31/78 

323 
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DATEl  »  02/25/78  -  30  days  «  01/25/78 

P  •  iromediaCe  predecessor  in  the  contract  to  DATEl  •  1  3/25/77 
S  a  immediate  successor  in  the  contract  to  DATEl  =  02/25/78. 


SUBiaT  OATS 

BCNP 

p 

10/25/77 

108 

DATEl 

01/25/78 

BCWPl 

S 

02/25/78 

205 

To  interpolate  the  missing  BCWPl  value,  a  set  of  ratios  is 
set  up  and  solved  for  BCWPl. 

DATEl  -  P  s  BGMP1.-,10,8 
S  -  P  205  -  108 

Solving  this  equation  for  BCWPl  provides  the  basic  algorithm 
for  interpolating  missing  values  in  the  database. 


BCWPl  ■  {Q1/2S/78  -  lQ/25/77)  *  (2Q5-1Q8)  +  108  -  180.75 

<02/25/78  -  10/25/77) 

The  sane  basic  fosmila  was  used  to  calculate  BCWP2,  BCWP3, 
ECWP4  and  BCWPS.  Similar  steps  were  taken  to  interpolate  the 
five  months  previous  data  for  each  submit  date  in  the 
database  for  ACWP  and  BCWS,  vdiich  eventually  allowed  for  the 
calculation  of  three  and  six  month  indexes  and  corresponding 
EACs. 
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The  script  in  Paradox  Application  Language  (PAL®)  used 
to  interpolate  the  values  for  one  month  prior,  two  months 
prior,  three  months  prior,  four  months  prior  and  five  months 
prior  to  each  report  date  of  data  is  provided. in  the 
remaining  pages. 


t 
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Apptt&dlx  C:  D«acriptiv«  Statistics 

This  appendix  contains  the  descriptive  statistics  used  to 
test  the  hypothesis.  Each  row  is  labeled  in  the  far  left  column 
and  the  column  entries  are  labeled  below  each  column  of  values. 


(JOl.lacly,  KlddX*  «r  Lata) 

MtJUl 

•S 

co«£  eC  Variatlw 

!!l  viT  uca  .  .  .  c»c  *»■*  ■cw»  *C1I» 


Each  page  contains  one  category  of  contracts.  The  All,  Early, 
Middle  and  Late  Stages  are  indicated  in  the  upper  left  comer  of 
the  far  left  column. 

The  MEAN  refers  to  the  average  percent  deviation  from  Cost  at 
Completion.  It  is  calculated  for  each  EAC. 


MEAN  %  deviation  from  EAC  ( ( (EAC-CAC) /CAC)  *100]  i/n 


The  first  row  of  values  in  the  table  are  the  average  percent 
deviations  from  CAC  for  each  of  the  nine  EAC  formulas,  plus  the 
average  values  of  CAC,  TAB,  BCWP  and  ACWP. 

The  SD  refers  to  the  Standard  Deviation.  The  SDs  are  the 
second  row  of  values  in  the  table. 

The  Coef  of  Variation  refers  to  the  coefficient  of  Variation. 
This  is  the  ratio  of  SD  to  mean,  presented  as  a  percentage.  The 
third  row  of  values  are  the  coefficients  of  variation. 

The  N  represents  the  number  of  contracts  in  the  category 
while  n  represents  the  number  of  reports  in  the  category. 

(Reports  are  subsets  of  contracts.) 


*The  TAB  is  replaced  with  BAC  for  the  page  with  Management  Reserve 
removed  from  the  calculations. 
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Appendix  D:  EAC  and  Index  Ceilino  and  FlQQr.Qraoha 


This  appendix  presents  graphs  of  EACs  and  their 
corresponding  indexes .  Each  page  contains  two  graphs .  ■  The 
top  graph  presents  EAC  average  percent  deviation  from  cost  at 
completion.  The  bottom  graph  presents  the  indexes  that 
correspond  to  each  EAC  in  the  preceding  graph. 

The  arzowe  on  the  horisontal  axes  (X  axis -Percent 
Complete)  represent  transition  points  between  Early,  Middle 
and  Late  Contract  Couplet ion  Stages.  Specifically: 

Early:  Contract  Percent  Complete  <  .35 
Middle:  .35  <  Contract  Percent  Complete  <a  .70 

Late;  Contract  Percent  Complete  >  .70 

The  first  six  pages  present  results  based  upon  each 
index  type  (cvimulative,  six-month  and  three-month)  .  The 
first  three  pages  represent  all  contracts  (cumulative,  six- 
month  and  three-month) .  The  next  three  pages  represent  all 
contracts  with  Management  Reserve  removed.  The  remaining 
graphs  present  the  EAC  ceiling  and  floor  and  their 
corresponding  indexes  for  each  category. 
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ALL  CONTRACTS  CUMULATIVE  EAC  &  INDEX  GRAPHS 


103 


ALL  CONTRACTS  SDC-MONTH  EAC  &  INDEX  GRAPHS 


♦— CPI3  - ■ - SPI3  - A - SCI3 


ALL  CONTRACTS  THREE-MONTH  EAC  &  INDEX  GRAPHS 


105 


16 


ALL  CONTRACTS  (MR  REMOVED)  SDC-MONTH  EAC  &  INDEX  GRAPHS 


ALL  CONTRACTS  (MR  REMOVED)  THREE-MONTH  EAC  &  INDEX  GRAPHS 


ALL  CONTRACTS  (MR  REMOVED)  EAC  &  INDEX  GRAPHS 


PRE-PRODUCTION  CONTRACTS  EAC  &  INDEX  GRAPHS 
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COST  PLUS  CONTRACTS  EAC  &  INDEX  GRAPHS 
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FIRM  FIXED  CONTRACTS  EAC  &  INDEX  GRAPHS 


CompUtlon 
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Completion 


NAVY  CONTRACTS  EAC  &  INDEX  GRAPHS 
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D«vl«llon  from  Coot  ol 
Complotlon 


USAF  CONTRACTS  EAC  &  INDEX  GRAPHS 
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Oavlatloii  from  Cost  at 
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ENGINE  CONTRACTS  EAC  &  INDEX  GRAPHS 
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